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ABSTRACT 



This document reports on a 6 -year study of science education 
reform, the Scope, Sequence, and Coordination (SS&C) Project. The reform 
included teacher enhancement activities as well as curricular materials and 
was designed to help science students achieve the National Research Council's 
Science Standards (1995) . The two components of the study were comparing 
students who had participated in the reform movement with those who had not, 
and investigating the long-term effects of the reform movement. Results 
indicate that overall, the reform effort was successful in making changes in 
science instruction. Recommendations for effective implementation through 
teacher enhancement efforts and for longitudinal evaluations of teacher 
enhancement projects are also included. (Contains 24 references and 
appendices with data tables.) (YDS) 
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EXECUTIVE SUMMARY 



This is the culminating report of an in-depth, six- year study of science education reform. The 
reform included teacher enhancement activities as well as curricular materials and was designed 
to help science students achieve the National Research Council’s Science Standards (NRC, 
1995). The longitudinal evaluation project was quite complex, used several data gathering 
methods and sources, and produced several reports and articles. The evaluation effort had two 
major components. The first component was designed to compare students who had 
participated in the reform effort with students from the same site who had not participated in the 
reform. The second component was to follow a subset of the sites to identify the long-term 
effects of the reform effort. For all six years of the evaluation effort both qualitative and 
quantitative data were gathered from principals, teachers and students through extensive site 
visits and assessment of student outcomes. The purpose of this report is to summarize and 
condense the findings from the subset sites. It presents the data gathered throughout the course 
of the evaluation effort by discussing the data from all of the sites as a set and by providing 
detailed information about each site individually. Furthermore the data are synthesized into a 
theoretical model for teacher enhancement and curricular implementation, and 
recommendations for future implementation and evaluation efforts are provided. 

Cross Site Results 

The classroom environments in the five schools changed significantly during the reform effort. 
There was more inquiry and more hands-on activity. After the funded reform effort ended, the 
classroom environments at the sites changed in different ways depending on the local context 
for the reform. Some sites continued or modified the changes while others moved back into 
more traditional methods. Consolidated impressions across the sites show that the sites were 
quite unique. The culture of each site interacted with the reform effort and played a significant 
role in the continuation, modification or regression of the reform. The culture of the site 
included several elements, such as the attitudes of the teachers toward change, the skills and 
philosophies of the teachers, the interactions of the teachers with the liaison teacher, the 
physical and emotional support available, the power structures within the school and external 
pressures, e.g., State or district mandates. 
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The continuing 9 th grade data showed changes year to year in classroom learning environments, 
student attitudes and student achievement. There were substantial differences among the five 
sites. There appeared to be only a weak relationship between what occurred in the classroom 
and student achievement although there was also little stability in what occurred in the 
classroom. There was a weak relationship between use of inquiry or laboratory activities and 
student achievement as measured by lab skills performance tests. The students in the reform 
effort showed slightly greater achievement when they participated in the reform for two years in 
a row. These increases were maintained somewhat in sites that continued in reform mode. 

The comparisons of the 12 th grade students who had participated in the reform with those who 
had not showed that the non-reform students performed slightly better on the science literacy 
multiple choice test while the reform students performed slightly better on laboratory 
investigations. The students who participated in the reform had slightly better attitudes toward 
science. The comparisons of changes in the 9 th and 12 th grade cohorts of students who had 
participated in the reform and those that had not, showed no consistent trends in the changes 
from 9 th to 12 th grade. 

The data showed that students with or without the reform were not achieving the standards at 
particularly high levels. Achievement on the science literacy multiple choice test was higher 
than on the more open-ended written items or on the extended laboratory investigation test. 
Planning controlled investigations appeared to be particularly difficult for the students. Students 
performed slightly better on planning investigations when they had equipment in front of them 
as opposed to when they imagined the experimental situation. 

Overall it appears that the reform effort was successful in causing changes in the way science 
was taught. The initial appearance and persistence of these changes, however, were dependent 
on the different elements present at the sites. Some sites and individual teachers gained and 
retained more from the reform effort than others. The institutionalization of change in a school 
was greatly affected by external pressures, the power structures in the school in relation to the 
reform, the availability of support and the desire for change. The data showed that if the same 
teachers continued to teach the reformed curriculum and worked on adapting it to match with 
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their environments, their students' achievement was more likely to be affected. Perhaps, as 
teachers become more experienced with reform, they become more comfortable teaching it, 
teach it more effectively, and convey clearer expectations for student performance. 

SS&C Model for Curricular Implementation 

The model for curricular implementation in Figure A was distilled from the SS&C experience. 

It is based on the notion that the intended curriculum is filtered and modified by a variety of 
different contextual factors. In the case of SS&C, the original curricular materials developed by 
NSTA were intended to be consistently implemented in all 13 sites across the country. In reality 
each site modified materials based upon the unique features of their school. 

The flow chart of curricular implementation in Figure A visually describes how the intended 
SS&C curriculum was first filtered through four inter-related factors: the community, the school 
and the students; the material and emotional support provided by the school; the teachers; and 
the liaison. Each site started with the materials developed by NSTA, but then considered how 
the materials could be used in their school with their own unique student population. The 
intended curriculum was also filtered through the support provided by the school and the 
existing resources available. The nature of the teachers, such as their content understanding, 
teaching philosophy, and openness to change, and of the liaison, such as his or her connections 
within the school power structure and facilitation skills, were also critical components of the 
curricular implementation. Ultimately, it was the teachers and the liaisons who made final 
decisions about the instruction and assessment that was used in the class. 

This resulted in the first enacted SS&C curriculum. This enacted curriculum was once again 
filtered through the same factors; however, this time around student outcome data and 
experiences with the curriculum were available. The teachers, parents and administrators had 
perceptions about the effectiveness of SS&C and these perceptions served as another filter for 
the curriculum. External pressures also impacted the SS&C curriculum. State mandated 
curriculum and assessment modified the implementation of SS&C at some sites. The result of 
this second filtering process was the implementation of the second enacted SS&C curriculum. 
Theoretically, the process could continue into the future. Each year the curriculum is modified 
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and reconsidered based upon student outcomes, the local context, and external pressures. The 
result is a cyclical curriculum implementation process that reconsiders and implements a 
modified curriculum each year. 



Figure A: SS&C Model for Curricular Implementation 
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Recommendations 



Recommendations for Effective Implementation Through Teacher Enhancement Efforts 

1 . Recognize that each school and teacher is unique and that consequently each 
implementation will also be unique. A project-wide curriculum will be modified by 
individual sites. 

2. Recognize that external pressures such as curricular and assessment requirements may 
significantly impact the new curricular implementation. 

3. Obtain district commitment to continue the implementation for at least three years 
with most of the same teachers. Effective change takes time. 

4. Obtain district commitment to provide the necessary facilities and materials as well as 
support a process for making the materials easily accessible. 

5. Involve teachers who either want to make a change or who are willing to try it and 
withhold judgment until later. Do not force the change on the teachers. 

6. Have small groups of the teachers who will be directly involved in the implementation 
attend training. 

7. Involve teachers in the development of the curriculum materials to help develop 
ownership and to help guarantee ease of implementation, but do not expect teachers to 
write the curriculum per se. The support of professional curriculum developers is 
needed in the initial stages of development. 

8. Provide personally supportive, local liaison teachers who have the knowledge 
necessary to be effective guides and who are part of the power structure. They can 
give the implementing teachers access to power and provide official support and 
sanction for their efforts. 
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9. Gather and provide evidence to the teachers and schools that the changes are 
producing valuable effects. 

10. Develop strong communication networks within and between sites, and between sites 
and the centralized project staff to increase teachers’ feelings of involvement and 
ownership in the project. The need for consistent and formalized avenues of 
communication increases as the number of implementation sites increases. 



Recommendations for Longitudinal Evaluations of Teacher Enhancement Projects 

1. Use the school or classroom as the unit of analysis with quantitative data and/or 
employ HLM techniques. 

2. Build rapport with the school and teachers to facilitate the data collection. Have a 
contact teacher in each school who can help evaluators meet data collection needs. 

3. Carefully document everything that is done throughout the project and develop 
comprehensive filing systems that allow back-ups of all vital materials. 

4. Select evaluation team members who will be able to continue to participate over the 
whole course for the study. 

5. Bring new people in and out of the evaluation team to provide new insights and 
energy. 

6. Have an external evaluation consultant to the evaluation effort who can review and 
refine the evaluation approaches. 

7. Produce publications or reports as the project progresses. These reports could be on 
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the main topic or involve more research-oriented special questions or interpretations 
about the data being collected. 

8. Include sufficient funding for strong clerical help in formatting reports, tracking data, 
editing, and numerous other administrative tasks. 

9. Be vigilant in the data analysis. Conduct routine crosschecks of data analysis 
procedures. Have more than one-person conduct analyses to serve as a crosscheck. 
Doing "side" studies can help to identity weaknesses in the analyses. 



I. INTRODUCTION 



This is the culminating report of an in-depth, six- year study of science education reform. The 
reform included teacher enhancement activities as well as curricular materials and was designed 
to help science students achieve the National Research Council’s Science Standards (NRC, 
1995). The reform effort was the Scope, Sequence and Coordination Project (SS&C) and 
consisted of two summer workshops, school year contact, and curricular materials matched to 
the instructional philosophy presented at the workshops. 

The evaluation effort had two major components. The first component was designed to contrast 
students who had participated in the reform efforts with students from the same site who had not 
participated in the reform. The second component was to follow a subset of the sites to identify 
the long-term effects of the reform effort. For the first three years data were gathered from all 
of the sites participating in the reform effort. For the next three years data were gathered from 
five of the original sites. For all six years both qualitative and quantitative data were gathered 
from principals, teachers and students. Student information included data from several different 
types of assessment measures in addition to questionnaires and interviews. 

This report summarizes the data gathered throughout the course of the evaluation effort in four 
ways. First this report provides detailed, longitudinal descriptions of how teacher enhancement 
operated across the five continuously studied sites. Second, it provides a theoretical model for 
curricular implementation grounded in the data that were gathered. Third, this report provides 
recommendations for future teacher enhancement efforts and their evaluations based on insights 
gained through the data. Finally this report provides detailed, longitudinal descriptions of how 
teacher enhancement operated in each of the five sites. 

Several other reports were produced as part of this evaluation effort. The initial report, Scope, 
Sequence and Coordination: 9 th Grade Science (October, 1996) contrasts the classroom learning 
environments, student attitudes and student achievement for 9 th grade students the years before 
and after the reform effort. This report also contains case studies of three teachers describing 
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their perceptions of their participation in the reform. The second report, Scope, Sequence and 
Coordination: 10 th Grade Science (September, 1997) provides information similar to the first 
report except it contains information on 10 th graders. The pattern of detailed reports providing 
yearly in depth single and cross site, 9 th grade information, for the five longitudinal sites was 
continued in two more reports: Evaluating the Long Term Effects of Teacher Enhancement 
(January, 1999) and Evaluating the Long Term Effects of Teacher Enhancement (June, 2000). 

Another report, The Supplementary Evaluation Report, Scope, Sequence and Coordination: 9 th 
and 10 th Grade Science (December, 1997) provides a comparison of the results for the 9 th and 
10 th graders and information on students who participated in the reform effort in both 9 th and 
10 th grade. This report shows that the students who participated in SS&C for two years were 
similar to all 10 th grade students with a few differences favoring the two-year SS&C students. 
Students who had taken SS&C for two years were more likely to say they would take science in 
12 th grade, and they had higher scores on physical science multiple choice items. 

The original cohorts of 9 th grade students were contacted again when they were in 12 th grade and 
asked to respond to a subset of the original evaluation instruments. The results contrasting the 
two cohorts over time are presented in the report entitled, Evaluating the Long Term Effects of 
Teacher Enhancement: 9 th and 12 th Grade Comparison (August, 2000). This report shows 
mixed results. Ten of the sixteen differences found in student motivation, participation in 
science activities outside of class, and attitudes toward science favor the SS&C students. In 
terms of achievement the comparison group students outscored the SS&C students on the 
multiple choice, open-ended, and lab skills performance tests while the SS&C students 
outperformed the comparison group students in conducting hands-on full investigations. 



In addition to the evaluation reports several journal articles and other publications were 

produced. These include: 

1. Huffman, D., & Lawrenz, F. (in review). Reforming science educationin urban 
schools. Journal of Research in Science Teaching. 

2. Lawrenz, F. & Huffman, D. (in review). Relationships among student, teacher 

and observer perceptions of science classrooms and student achievement. Educational 
Evaluation and Policy Analysis. 

3. Huffman, D. (in review). Evaluating science inquiry: A mixed- method approach. 

In J. Altschuld & D. Kumar (Eds.) Program Evaluation in Science Education 
The Netherlands: Kluwer. 

4. Lawrenz, F., Huffman, D. & Welch, W. (in press). The science 
achievement of various sub-groups on alternative assessment. Science 
Education. 

5. Huffman, D., Sinclair, M. & Lawrenz, F. (in press). The relationship between 
the science classroom learning environment and achievement among students 
with and without disabilities. Journal for Elementary/Middle Level Science 

Teachers. 

6. Lawrenz, F., Huffman, D. & Welch, W. (2000). Policy considerations 
based on a cost analysis of alternative test formats in large scale science 
assessments. Journal of Research in Science Teaching, 37(6), 615-626. 

7. Huffman, D. & Lawrenz, F. (Fall, 1999). Achieving the science standards: A 
national study of inquiry-based instruction in high school science. In D. 

Huffman (Ed.), Research Practice (pp. 17-24). Center for Applied Research & 
Educational Improvement, University of Minnesota. 

8. Welch, W., Lawrenz, F. & Huffman, D. (1998). The precision of data 
obtained in large scale science assessments: An investigation of 
bootstrapping and half- sample replication methods. Journal of Research in 
Science Teaching, 35(6), 697-704. 

9. Huffman, D., Lawrenz, F. & Minger, M. (1997). Within-class analysis of 
ninth-grade science students' perceptions of the learning environment. 

Journal of Research in Science Teaching, 34(8), 791-804. 

10. Aldridge, B., Lawrenz, F. & Huffman, D. (1997). Scope, Sequence & 

Coordination: Tracking the success of an innovative reform project. 

The Science Teacher, 64(1), 21-25. 
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11. Lawrenz, F. & Huffman, D. (1995). The SS&C project: Project evaluation and 
national science education standards. In D. Tulip & D. Smith (Eds.), Conference 
Proceedings of the Australian Science Teachers Association Queensland, 

Australia: Queensland University of Technology. 

In addition to the information and charts presented in this report, an Appendix of Tables has 

been compiled. The Appendix contains tables that support each chart found in the report. The 

Appendix also contains additional tables, which provide more detail about specific results. 
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II. HISTORY 



In order to understand the context of this evaluation project and the reform effort it investigates, 
it is important to know the history. The reform effort was built on a vision of science education 
developed by Bill Aldridge, former executive director of the National Science Teachers 
Association, and his associates. The vision was formed out of the concern for the United States’ 
leadership in science and mathematics prompted by the international comparisons of student 
performance in mathematics and science. The vision was embodied in the title, “Scope, 
Sequence and Coordination." Scope meant that the studies should be rigorous. Sequence meant 
that the students should have experiences first and then tie these experiences to concepts. 
Coordination meant that each science was to be coordinated with the others, not presented one 
at a time. Although there was a middle school SS&C project, this report is about the high 
school project. The project was headed by Aldridge and controlled centrally with all sites using 
the same curriculum materials. The goal of the project was to improve student performance on 
the newly developed National Science Education Standards (NRC, 1995). 

The Intervention 

The National Science Teachers Association (NSTA) believed there was a need for a new 
coordinated approach to science education because so many students were underachieving in the 
sciences. A coordinated approach where every science is taught every year was advocated 
because so many students are systematically filtered out of the advanced sciences. Essentially, 
students identified at an early age as the “most able” are encouraged to take advanced science 
courses, while “less able” students take only the minimum amount of science. Over 50% of the 
students in the U.S. only take one year of high school science. Minority students, who tend to 
be predominately enrolled in urban schools, are even less likely to enroll in advanced science 
courses than other students (Mullis et al., 1994). There is a belief that other countries tend to 
educate the elite in the sciences, while the U.S. teaches to all students; however, in the sciences 
quite the opposite is true. The data suggest that the U.S. filters “low achieving” students out of 
the sciences. The NSTA believed that part of the reason students fail to pursue the sciences is 
that in the U.S. the curriculum tends to be “layer-caked.” Typically, physical science is taught 
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in 9 th grade, biology in 10 th grade, chemistry in 1 1 th grade, and physics in 12 th grade. Coupled 
with graduation requirements of only one or two years of high school science, this approach 
tends to lead to a less in-depth understanding of the sciences, especially in chemistry and 
physics. Most European and Asian countries, which also tend to score higher than the U.S. on 
international comparisons of science achievement, tend to use a coordinated approach to science 
where every science is taught every year. The NSTA sought to develop such a coordinated 
science program that would allow all students to study all sciences every year, instead of 
reserving advanced science for only advanced students. 

The ultimate goal of the project was to help students achieve the National Science Education 
Standards (NRC, 1995). The NRC Science Education Standards were developed as guidelines 
for achieving science literacy for all students, and include concepts and skills that all students 
should achieve by the end of high school. Although the NRC Standards provide excellent 
guidelines for science teaching, they do not provide specific lessons or procedures for teachers, 
and they were not designed to be an implemented curriculum. Translating the Standards into 
actual classroom practice still remains to be accomplished. The NSTA project was one of the 
first comprehensive effo rts designed to translate the Science Education Standards into actual 
classroom practice. It was essentially an inquiry- oriented science program designed to help all 
students achieve the science standards. 

The basic tenets underlying the project were: 

1) every student should study the four science subjects of biology, chemistry, physics and 
the earth/space sciences every year; 

2) science teaching should take into account students' prior knowledge and experience; 

3) students should be provided with a sequence of content from concrete experiences and 
descriptive expression to abstract symbolism and quantitative expression; 

4) students should be provided with concrete experiences with science phenomena before 
the use of terminology that describes or represents those phenomena; 

5) concepts, principles, and theories should be revisited at successively higher levels of 
abstraction; 
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6) learning should be coordinated in the four science subjects so as to interrelate basic 
concepts and principles; 

7) teaching should utilize the short-term motivational power of relevance by connecting the 
science learned to subject areas outside of science, to the practical applications of how 
devices in our technology work, and to the challenge of solving those personal and 
societal problems that have relevant underlying scientific components; 

8) teaching should utilize the long-term motivational power of sudden and profound 
understandings of science and of the awe which stems from comprehension of the power 
and universality of a relatively small number of fundamental principals of science; 

9) coverage of topics should be greatly reduced with an increased emphasis on greater 
depth of understanding of those fewer fundamental topics; 

10) assessment methods, items, and instruments to measure student skills, knowledge, 
understandings, and attitudes should be consistent with 1-9. 

The NSTA project included both curriculum development and teacher enhancement. In fact, 
NSF jointly funded the project under both the Curriculum De\elopment and Teacher 
Enhancement divisions. The curriculum was developed by bringing together high school 
teachers from around the country to design activities and leam how to implement the activities 
using an inquiry- oriented pedagogy as defined by theNRC National Science Education 
Standards. Site-based implementation teams were formed to actually implement the curriculum 
in the classroom, and follow-up workshops were held throughout the school year to work on 
modifications to the curriculum. The 9 th grade curriculum was first implemented in the 1995-96 
school year. 

The SS&C project was designed to provide teacher enhancement and curricular materials 
through national workshops and committees. During the first year of the project the teachers 
were to develop, pilot test and revise lessons that fit the ten tenets of SS&C. These lessons were 
to serve as a basis for an intensive summer workshop where the teachers would be steeped in the 
SS&C philosophy and work as a development team in the finalization of the draft 9 th grade 
SS&C curriculum. The idea was that this involvement in the development process would allow 
the teachers to construct a common meaning of the SS&C philosophy so that they would “own” 





it when they returned to their classrooms. The teachers were also members of various national 
committees, e.g., assessment, where they worked on specific ideas to be shared with the whole 
group. The plan was that the next year and summer would focus on the development of 10 th 
grade, the next on 1 1 th grade and finally in the fourth year 12 th grade. 

There were thirteen schools participating in the project originally. Two in North Carolina, three 
in Texas, three in California, two in Iowa and one each in Montana, New York and Washington, 
DC. Several of the States: California, Iowa, Texas, and North Carolina, had participated in the 
middle school project and were familiar with the philosophy of SS&C and had some 
infrastructures in place for support of the teachers. In these states local liaisons, generally 
affiliated with a local university, provided guidance to the teachers in the schools and offered 
workshops throughout the school year to further the teacher enhancement. All sites had within 
the school liaisons that helped other teachers on a one-on-one basis to implement SS&C. 

After the first year and half, the continuation of the project was questioned. There were 
concerns about the cost of the project, the quality of the curricular materials and the ability to 
move the project into the 1 1 th and 12 th grades. NSF originally funded the project to develop 
only 9 th and 10 th grade materials. More funding was needed to develop 1 1 th and 12 th grade 
materials. There was no official word on continuation of funding until after the second summer 
workshop, so that workshop continued as planned. The purpose of the second summer 
workshop was to provide opportunity for enhancement for the 10 th grade SS&C teachers. This 
included some of the teachers who had participated in the 9 th grade instruction and enhancement 
as well as new teachers. After the summer workshop the project received word that it had not 
been funded to continue development into the 1 1 th and 12 th grades. There were some funds left 
to finish out the school year but after that the project would have to continue through the efforts 
of the individual sites. This was quite a blow to the participating schools, but since the school 
year was beginning and the curricular materials were ready, they went ahead with offering the 
10 th grade SS&C. 

The longitudinal evaluation of SS&C was summative in nature. It was distinct from the reform 
effort in that it operated at a different site, was conducted by a different PI and had its own 
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separate advisory committee. The evaluation used a multi- method, time lag, quasi- experimental 
design where the students in each school during the first year when the reform was not being 
implemented served as the comparison group for students in the subsequent years when the 
reform was implemented. In this way most of the contextual variables were controlled. The 
evaluation gathered data on three major outcomes: student achievement, student attitude and 
motivation for science, and classroom activities. Evaluation team members monitored 
classroom activities through observations, student and teacher questionnaires, and student, 
teacher and principal interviews. Student attitude and motivation were measured through 
questionnaire data. Student achievement was measured through the use of several different 
types of assessments. Students completed multiple choice items, open-ended items, laboratory 
stations and full scale experiments. Case studies of individual teachers were also conducted to 
shed light on their perceptions of the reform process. 

The evaluation began as soon as SS&C was funded in order to gather comparison year data. 

Data the first year were collected from 9 th grade students who had not participated in the reform. 
Data the second year were collected from 9 th grade students who had participated in the reform 
and from 1 0 th grade students who had not (the prior year 9 th graders were re- tested). The news 
about the discontinuation of funding was obtained as the evaluation was preparing for the third 
year of data collection which would be to collect a second year of reform data from 9 graders 
and a first year of reform data from 10 th graders. The evaluation continued mostly as planned 
for that year with remaining funds. 

Because the evaluation results were informative and showed site based effects, the evaluation 
team felt that it would be important to continue their examination of the impact of the reform 
effort. Therefore the evaluation team proposed a new evaluation project that would examine the 
effects of the reform effort in a more longitudinal and process oriented fashion by following five 
of the original thirteen sites. The purpose of the evaluation was to better understand what 
happened to a major teacher enhancement effort over time. The same comprehensive data 
collection continued and was augmented by more in-depth interviews and more questions about 
the reform process. This extended evaluation project also gathered data from the students in 
12 th grade that had been tested in 9 th and 10 th grade. 
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III. DATA COLLECTION 



There were two phases of data collection. Phase one was for the first three years of the project 
and included thirteen schools, and phase two was for the last three years and included five 
schools. This report concentrates on the case studies of the five schools including data from all 
six years. A longitudinal, mixed- method case study approach was used. The study was mixed- 
method in that it attempted to integrate both case study and comparison approaches as described 
by Green and Caracelli (1997). The case studies purposefully included a mix of qualitative and 
quantitative data in order to examine the impact of the NSTA curriculum from multiple 
perspectives (Yin, 1994). The case study was comparative in nature, meaning that students in 
traditional science classes (the 9 th grade students in the 1994-95 school year) were compared 
with students in subsequent years who used the new NSTA coordinated science approach (9 th 
grade students in the 1996 - 2000 school years). In essence, it was a time-lag design where the 
prior year's science students were compared to each subsequent year's science students. 

The continuing case study schools were chosen from the thirteen schools involved in the first 
three years of evaluation of SS&C. They were selected to be representative of the diversity in 
the U.S. in terms of geographic areas, population and race/ethnicity. The schools were located 
in California, Iowa, Montana, New York, and Texas. There were approximately 20 teachers 
teaching 9 th grade science classes at these schools and over 1 ,500 students being taught by these 
teachers each year. Ninth grade science courses at these schools during the 1994-95 comparison 
school year had been quite diverse; some courses concentrated on a single science area such as 
physical science, life science or earth and space science, while other courses included various 
arrangements of the sciences such as integrated and coordinated science. Some courses were a 
year long and met everyday for 45 minutes, while other courses were only a semester long and 
met 90 minutes each day. 

The evaluation team developed ten instruments. Because the study was based on the premise 
that the learning environment affects student achievement, both the learning environment and 
student achievement were assessed. Contextual variables were assessed using a student 
questionnaire, a teacher questionnaire, a classroom observation schedule, principal, teacher and 
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student interview protocols, and a course content survey. Because the major measure of the 
effectiveness of the SS&C project was student achievement, it was examined from several 
different perspectives. Students participated in a five station hands-on laboratory skills test, 
answered multiple choice science content items, designed and conducted an experiment, and 
answered open-ended science content items. Standardized administration and scoring protocols 
were developed to ensure consistency and objectivity. Descriptions of all assessment 
instruments, the instrument development process, and the psychometric properties of the 
instruments are included in the Evaluating the Long Term Effects of Teacher Enhancement 
(February, 2001) Appendix. 

A nested data collection design was used to gather information at each of the school sites. Data 
were collected from all participating 9 th grade students and teachers. In addition, three 9 th grade 
classes at each school were targeted for more comprehensive data collection. Except for the 
science literacy test and the course content survey, which were mailed to the schools near the 
end of the school year, the data were collected by the evaluation team during site visits. Site 
visits were conducted in the spring and fall during each of the six years of the study, except the 
fall of 1996. During the fall and spring visits, all 9 th grade teachers were interviewed and the 
three target classes were observed. During the spring visit, in addition to the interviews and 
observations, most of the 9 th grade students were given a questionnaire to complete. Those not 
taking the questionnaire were six to twelve of the students in each target class who were 
randomly selected to take the lab skills performance tests. One student from each target class 
was also interviewed. The school principal was interviewed only during the fall site visit. See 
Figure B for a schematic of the data collection instruments and sources of information. 



Figure B: Data Collection Instruments and Sources of Information 




In addition to collecting data from 9 th grade teachers and students at the five schools for the 
entire six years, data were collected from 10 th graders for two years (one comparison group and 
one SS&C group) and from 12 th graders for two years (one comparison group and one SS&C 
group). 

The data collection procedures and instruments for 10 th grade data collection were identical to 
the procedures and instruments described above for the 9 th grade. For the 12 th grade data 
collection, the teacher interview, classroom observations, and course content survey forms were 
not used since 12 th grade students were not all enrolled in science classes. 12 th grade 



O 

ERIC 



19 



24 



achievement assessment differed from 9 th grade assessment as well: 12 th grade students 
answered a shortened version of the science literacy test, designed and conducted one of three 
experiments and were not asked to answer any open-ended science content questions. Two 
reports, Supplementary Evaluation Report, Scope, Sequence & Coordination: 9 th and 10 th Grade 
Science (December, 1997) and Evaluating the Long Term Effects of Teacher Enhancement, 9 th 
and 12 th Grade Comparison (August, 2000), fully describe the data collection procedures and 
instruments used to collect these data. 

The results of the tests and questionnaires were analyzed using the students as the unit of 
analysis. In evaluation reports of the first three years, the class was used as the unit of analysis 
because data from all 13 schools were combined and there were over 100 classes in the 
evaluation. The present study is a case study approach, and the data from each school are not 
combined. There were not enough classes at each school to analyze the data by class, and 
therefore the data were all analyzed using the student as the unit of analysis. 
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IV. CROSS SITE FINDINGS AND IMPRESSIONS 



This section covers a variety of topics related to cross site findings over the six years of the 
study. First it presents brief summary information about the persistence of the reform in all of 
the 13 original sites. This is followed by capsule descriptions of the five case study sites. After 
these descriptions, consolidated cross site data about the five sites is presented in terms of 
learning environments, student attitudes and motivation, and student achievement. The section 
concludes with three different syntheses of the data: descriptive impressions, a diagrammatic 
model and recommendations. The consolidated cross site data is presented both textually and in 
charts. The charts in this section show trends over the six years of the study for each of the five 
followed sites. The data displayed in the charts are also presented in detail in the tables in the 
Appendix. 

In this study, the scores on all of the measures administered in the years after the reform was 
implemented (1996, 1997, 1998, 1999, 2000) were contrasted with the scores on the measures in 
the comparison year (1995). The contrasts were conducted using various statistical tests, and p 
levels of .05 or less determined significance. Such changes are referred to as "statistically 
significant" or "significant" within the text. Additionally, each student outcome cross site chart 
has a corresponding effect size chart. Effect size is defined as the measure of the degree to 
which the mean of one population and the mean of another population differ in terms of the 
standard deviation of the parent population. Because of the large differences between the years, 
the standard deviation was calculated for each pair of years separately. The measure of standard 
deviation used was the " square root of the pooled variance" for the two years being compared 
for each instrument. This procedure best controlled for statistical bias and error. The formula 
used was: d=Meanl -Mean2/square root of pooled variance within group. The effect size charts 
allow us to consider relative educational significance in addition to statistical significance. For 
the purposes of this report, an effect size of ± 0.5 was considered to be educationally significant 
(Howell, 1997). 
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Persistence of the Reform 

Detailed descriptions of all 13 of the sites that initially participated in the teacher enhancement 
cannot be provided because only five of the original 1 3 were followed over time. However, in 
the spring of 2000, seven of the original eight sites that had not been followed were contacted to 
determine what aspects of the SS&C reform remained at their schools. The eighth school was 
not contacted because it had never really implemented SS&C. The researchers spoke with one 
teacher involved in SS&C at six of the seven schools that were contacted. One school did not 
have anyone remaining on staff that had been involved with SS&C. This information was 
combined with the data from the five case study sites to provide this snapshot of persistence of 
the reform. 

At the time of the interview, three of the seven schools were no longer using any SS&C 
curricula or activities. Three schools were using modified SS&C activities and curricula, and 
one school was using only the earth and environmental science SS&C activities. Similarly, two 
of the five case study sites were still using the curriculum. One of the case study sites was using 
it in a significantly modified fashion, and two were not using it at all. 



Level Of SS&C Curriculum 1 


Jse After Five Years 


Never 

Used 


Stopped No 
More Use 


Significant 

Modifications 


Substantially 

Still 

Use 


1 


5 


5 


2 



When asked what SS&C teaching techniques were still being used at their school, most of the 
seven non case study schools indicated their schools were using an activities based program, 
hands-on activities, inquiry-based science or an integrated approach. One school indicated they 
might be using SS&C activities and techniques but no faculty familiar with SS&C remained to 
identify them as such. Two of the teachers pointed out that their schools used SS&C teaching 
techniques but not as a result of being involved with SS&C. At the case study sites the two sites 
which had continued to use SS&C also continued to use SS&C teaching techniques, as did the 
site which had continued to use SS&C with modifications. One of the sites that was not using 
SS&C was using the techniques, but was using them because they had been in use before 
SS&C. At the remaining site only one teacher was using the techniques. 





When asked about use of the type of assessment techniques recommended by SS&C three 
teachers reported that none were being used at their school. At the remaining four schools, one 
teacher indicated the teachers used portfolio assessment, fewer paper and pencil assessments 
and students made PowerPoint presentations. Another school used some performance 
assessment but used more objective testing than SS&C assessments. The teachers from the third 
and fourth schools said they used some of the SS&C assessment ideas. At the five case study 
sites, four were using some assessments of the type recommended by SS&C and one was not. 

When discussing the SS&C philosophy that still remained at their schools, the teachers most 
often mentioned a commitment to hands-on learning. Two schools in one State reported being 
in support of integrating the curriculum but that the State had mandated that they do not 
integrate. Three teachers indicated that the integration philosophy remained. Two schools in 
another State reported that none of the SS&C philosophy remained, one citing that the textbook 
dictated philosophy and the other citing the complete turn over in science faculty at the school. 
The situation was similar at the five case study sites. The philosophy remained at the three sites 
using at least parts of the curriculum. One of the remaining sites had an inquiry-based 
philosophy before SS&C, and at the fifth site only one teacher retained any of the philosophy. 

The Case Study Sites 

The continuing contact with the five case study sites, including in-depth discussions with the 
participants and extensive data collection, provided rich sources for reflection on elements 
necessary for successful teacher enhancement. Because it is difficult to understand 
generalizations without any knowledge of the individual entities, capsule descriptions of each of 
the five sites are provided here. More detailed information is included in the by site sections of 
this report. 

1 . This Montana site was a large, mostly Caucasian junior high school in a fairly large rural 
city. The school had the necessary facilities and materials to support the curriculum and had 
few behavior or discipline problems. The 9 th graders were given the choice of participating 
in the curriculum, and enrollments were high. Two 9 th grade teachers attended the national 
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training, and these same two were personally responsible for the 9 th grade curriculum. Prior 
to involvement in the project the teachers had felt successful with their students but they felt 
this new coordinated approach was the "way to go" in order to have all students study all the 
science. After the training the teachers were committed to the new philosophy and believed 
it would improve student learning. An outside, non 9th grade teacher served as the SS&C 
liaison person, facilitated their implementation of the course and its philosophy, and 
provided personalized encouragement to the teachers. No new teachers were required to 
teach the curriculum until several years after it had become well established. When new 
teachers were assigned, the former teachers guided them. The curriculum with some 
modifications continued throughout the six years of the study period. 

2. This California site was a large multi-ethnic high school in a very large city. The school 
facilities were not always adequate for the needs of the curriculum, but the teachers worked 
hard to gather the necessary materials. The student body was more apathetic than 
problematic. All 9 th grade students participated in the curriculum. Several teachers were 
responsible for the teaching of 9 th grade science. Different subsets of these teachers attended 
the national training. Over half their students were failing 9 th grade science, and they 
believed SS&C would help more students succeed in science. After the training the teachers 
worked as a group to develop the specific lessons for their school and met often to keep 
everything organized. They had a liaison teacher who taught 9 th grade sometimes and who 
supported their efforts. The team supported any new teachers as they moved into teaching 
9 th grade. In order to better match a statewide testing program, the curriculum was modified 
after the first two years. This modified curriculum remained in place until the end of the 
study. 

3. This Iowa site was a mostly Caucasian, medium-sized high school in the first ring suburbs 
of a moderately sized city. The school had sufficient resources to support the curriculum 
and large numbers of the students went on to higher education after high school. All 9 th 
graders (except a small group of honors students) participated in the curriculum. Two 
teachers attended the training, and this same group was in charge of the teaching of 9 th grade 
science for some time. They were experienced teachers and believed they were doing well, 
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but were willing to give the new philosophy and methods a chance, in order to give their 
students a better and more diversified science experience in 9 th grade. The liaison teachers 
(one retired) were senior members of the science department who strongly supported the 9 th 
grade teachers. One of the originally trained teachers came to "own" the curriculum, and 
she helped new teachers to understand the techniques and philosophy. Much of the initial 
SS&C curriculum remained in use during the course of this study, but it was continually 
modified and revamped to better fit what the teachers believed was necessary for their 
students. 

4. This Texas site was a large, mostly African American high school in a very large city. The 
school facilities were not adequate to support the curriculum, and although teachers made 
efforts to obtain the necessary materials, they were generally not successful. The students 
were very polite although not particularly engaged in academics. All 9 th grade students took 
SS&C. There was a group of three experienced teachers who attended the national training 
and some others who attended local training. A large group of teachers taught 9 th grade over 
the course of the project because of high teacher turnover. For the first two years the new 
teachers were encouraged to teach using SS&C philosophy and lessons by the liaison 
teacher who was also the science area chair, but the liaison teacher was not part of the 9 th 
grade team. She stopped being chair after the first year and retired. In the third year the 
State mandated a new course for 9 th grade, so the SS&C curriculum was not used after that 
although some of the SS&C activities that fit into the new course were used by some of the 
teachers. 

5. This New York site was a large ethnically mixed high school in a suburban area. The school 
had excellent facilities to support the curriculum. Only a subset of the students took the 
SS&C course, and most of these were not college bound. Several teachers attended the 
national training, and a large group of teachers was involved in teaching the 9 th grade SS&C 
course because a different set taught it each year. The teachers in the school already had a 
hands-on, inquiry approach to science so the SS&C philosophy matched theirs. The 
teachers at the initial training adapted the curriculum to their needs and implemented that 
modified version. The liaison teacher was “in charge” and significantly encouraged the 
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teachers to implement the program. The State mandated a new course for 9 th graders in the 
fourth year of the project, so the SS&C curriculum was not used after that. 



Learning Environments 

Despite the different elements affecting the teacher enhancement efforts at the schools, the 
project appears to have had a significant impact on the classrooms. The philosophy of SS&C, 
like that of the national standards, is one that calls for student-centered learning communities 
where students design and conduct their own investigations. For the first two years of the 
enhancement all of the sites but one showed significant changes from the comparison year in the 
types of activities used in science classrooms. All sites became much more student- centered in 
their instruction focusing on hands-on activities, group work, experimentation, the presentation 
of data or activity before the introduction of terms or concepts — precisely what the enhancement 
called for. 

The following two scenarios capture some of the differences between comparison and SS&C 
science classes during the first two years. 

Comparison Scenario: 

Mr. Watkins had a good earth science lesson planned for the day and he knew the students 
would enjoy it. The students would be working on an experiment and he had all his 
materials placed conveniently on the lab tables. Mr. Watkins stood in the doorway and 
welcomed the incoming students and when the bell rang he took attendance. He began the 
class with a rhetorical question framing the day's activities which he then answered using 
overheads and models. He asked several students questions recalling previous earth science 
material that supplemented his presentation. After presenting the concept, he passed out the 
lab sheets and had one of the students read it to the rest of the class. He commented on any 
difficult parts and demonstrated procedures as the students followed along. When all 
questions were answered the students moved to the lab area and conducted their experiments 
sharing equipment in pairs. As the students completed the lab work, they went back to their 
seats and answered the questions on the lab sheet. Mr. Watkins then went over the 
questions with the whole group calling on students to read their answers. He reviewed the 
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vocabulary and pointed out how the lab results confirmed the concept he had presented 
earlier. He then had the students re- state the concept in their own terms and ended the class. 

SS&C Scenario: 

Ms. Moore was looking forward to her class. There was an interesting earth science 
question to investigate and she was not sure what the students would find out. She had sets 
of materials out on the back shelf that the students could use. She welcomed the students as 
they came in and had them sit in their lab groups. She stated the topic for today's class and 
asked the students what they knew about it and if anything from the life science portion of 
the class related to this topic. Finally, she asked the students a question and asked how they 
might go about finding the answer. After listening to a few suggestions, she directed the 
students to use the materials to determine an answer. The students worked in their groups 
with Ms. Moore helping out as necessary when the group members could not answer their 
own questions. When all the groups had finished collecting their data, Ms. Moore asked 
them to report to each other what they had found out. The groups shared their results and 
their thoughts on answers to the lab question. There was a disagreement between two of the 
groups, which Ms. Moore suggested they could check on by doing some more lab work the 
next day. The students left the class debating their answers. 

There were costs and benefits to these changes to the classroom environment. The student 
interviews and opinionnaires showed that the students were quite satisfied with their classes. 
The students thought the classes were fun, not too hard and that the best things were the 
activities. The teacher interviews, however, revealed their mixed feelings about the 
implementation effort. All of the teachers felt somewhat overwhelmed by the SS&C curricular 
requirements. In the first years the materials were being developed/revised as the teachers were 
using them so there was not a "tried and true" way to go with all the glitches highlighted and 
compensated for. This lack of organization did, however, tend to promote individual 
modification and adaptation to the sites' needs. In addition to organizing and selecting 
curricular materials, the teachers felt they had to spend unreasonable amounts of time preparing 
hands-on laboratory materials for the students to use. Furthermore teachers often reported 
having to buy materials on their own or having to use materials that were not quite right. 
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Although the teachers thought the SS&C hands-on approach and having the students use the 
materials before the concepts were presented were good ideas, they felt it was just too difficult 
to do as often as the curriculum recommended. 

After the first two years of implementation the five sites that were followed longitudinally 
moved in different directions and the learning environments changed in accordance with these. 
The following charts show the movement over time in terms of the classroom environment 
measures. Although classrooms were observed and both students and teachers were asked about 
what activities occurred, the best predictor of student achievement was observation, so only 
those results are presented in this section. All data are presented in the by site sections of this 
report. The observation forms required coding classroom activity every five minutes. These 
codes were grouped into the student-centered, teacher- centered and, student and teacher- 
centered categories presented in the following charts. 

Individual sites showed unique results, but in general, all the sites but one, showed significant 
increases in student -centered activities during the first years of implementation with 
corresponding decreases (not necessarily significant) in teacher-centered activities. Student- 
centered activities went from being used about 10-25% of the time to 40-90%. During the 
fourth year there were increases in student- and teacher-centered activity and some decreases in 
student -centered activity with Iowa and Montana remaining higher than the comparison year 
and with the other sites returning to comparison year levels. California seemed to lag a year 
behind the other sites. The two sites, which substantially continued with the reform, Montana 
and Iowa, showed the most gain and consistency in student-centered activities. 
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Figure 1 

Classroom Observations 
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Figure 2 
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Figure 3 

Classroom Observations 
Percent of Time Spent in 
Teacher- and Student-Centered Activities 
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Attitudes and Motivation 

All of the 9 th grade students at the five case study sites completed questionnaires that contained 
several items related to their attitudes and motivation toward science. A combined score from 
these items is presented here. The by site sections of this report contain charts displaying less 
aggregated results, and detailed results by item are included in the Tables in the Appendix. 



The motivation, inclination and attitude data were combined to make one overall score for each 
school, each year, which served as an overall measure of affect. A score of “1” was assigned 
when the students answered “Yes” (they were inclined to take science or they were motivated). 
A score of “0” was assigned when the students answered “No” or negatively to the questions. 
The attitude questions were not “Yes/No” questions so they were re-coded as positive or 
negative attitude responses so that they could be combined with the inclination and motivation 
data. Surprisingly both the lowest and highest percentages of students saying “Yes” occurred in 
one site, Iowa, where the percentages ranged from 54% to 79%. Given this range it is clear that 
the majority of the students from all sites found science motivating. 
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In terms of the implementation, it would seem logical that the most immediate effect of more 
stimulating classroom learning environments would be improved student attitudes toward 
science. That, however, only occurred to some extent. In the first year of SS&C students at 
Texas and New York viewed their classes as significantly more motivating than the students had 
in the comparison year. In the second year the students at Iowa, Montana and Texas reported 
their classes as significantly more motivating, and this heightened level of motivation was 
retained in Texas for the third year and in Iowa throughout the course of the study. California 
showed significant decreases in all years but two. The effect size data for Iowa and Texas 
showed positive educational significance in several years as well. New York showed a positive, 
educationally significant effect size in the first year and a negative, educationally significant 
effect size in the last year. 



Figure 4 

Inclination, Motivation & Attitudes 
Student Means 

Percentage of Students who replied "YES" 
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Figure 5 

Inclination, Motivation & Attitudes 
Effect Size 
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Achievement 

All of the 9 th graders at the five sites completed measures of achievement. All students took the 
multiple choice and open-ended items, and randomly selected students completed the laboratory 
tests. The following charts show the movement over time in terms of the achievement 
measures. 

The achievement results were mixed. There were initial and continuing significant positive 
changes in the students' scores on the laboratory measures but not on the paper and pencil, 
multiple choice and open-ended items. These quantitative findings were supported by the 
teachers' opinions. Teachers at several of the sites believed students who had taken SS&C were 
better prepared to conduct inquiry science. However the teachers also believed that the students 
needed more help to understand the concepts. They felt the students needed the concepts 
verbally explained to them and that the SS&C sequence of teaching by inquiry resulted in the 
students not really understanding the concepts and not learning the necessary facts and 
vocabulary. Teachers also felt the students did not get enough depth in each of the sciences. 
They felt the students got a little bit of earth science and then were off to something else in life 
science or physical science. 
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The lab skills performance test data showed fluctuation across the sites. The general pattern 
was higher scores after the comparison year. The range of percent correct was 41 (Texas) to 82 
(Montana). In the years of the study there were 1 1 statistically significart differences in scores 
from the comparison year. In all but one year the Montana and New York scores were 
significantly higher than the comparison year score. The scores for Texas went back and forth 
between significantly higher to no difference. Iowa showed a significant increase only in the 
last year but also showed a consistently increasing trend away from the comparison year over 
the time period. California showed no significant differences. The effect size chart shows that 
Montana (1996, 1997, 1999 and 2000), New York (1996-1998 and 2000), and Texas (1996, 
1998 and 2000) had positive educationally significant effect sizes. California had a negative 
educationally significant effect size in 1997. 



Figure 6 

Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score 

California New York 

Iowa - - - jexas 

“ ■ ■ Montana 




Comparison 1995 SS&C 1 1996 SS&C21997 SS&C31998 SS&C41999 SS&C 5 2000 



ERIC 



33 



33 



Figure 7 

Lab Skills Performance Test 
Effect Size 
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The science literacy multiple choice achievement test scores are almost the mirror image of the 
lab skills performance test scores. California showed a significant positive change while the 
other four sites showed no change or significant negative scores. The range of the percent 
correct was 33 (Texas) to 67 (Iowa). Iowa scores were significantly lower than the comparison 
year in all years; Texas scores were significantly lower than the comparison year in 1997, 1999 
and 2000. California had significantly higher scores in three of the years. The effect size chart 
shows that negative educationally significant changes occurred for Iowa during three years and 
for Texas the last year. 
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Figure 8 

Science Literacy Test 
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Figure 9 
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The results on the science literacy open-ended measure were inconsistent with most scores 
wavering back and forth at approximately the comparison year levels. The range of percent 
correct from the science literacy open-ended questions was 10 (Texas) to 56 (Montana). New 
York (1996, 1997, 1998 and 2000) and California (1998) had scores significantly higher than 
the comparison year. Iowa (1996, 1999 and 2000), Texas (1999 and 2000), and Montana (1998 
and 2000), had scores significantly lower than the comparison year. The effect size chart shows 
positive educationally significant changes in New York (1996-1998 and 2000) and California 
(1998). Iowa in 1996 and Texas in 1999 and 2000 had negative educationally significant 
changes. 



Figure 10 
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Figure 11 

Science Literacy Test 
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1995-1995 1995-1996 1995-1997 1995-1998 1995-1999 1995-2000 

Consolidated Impressions 

Although the sites were diverse, it was possible to form some general impressions, which 
follow. These impressions were also synthesized into the theoretical model presented in the 
next section and formed the basis for the recommendations. 



The most important impression was that each school was indeed a unique community. There 
were, of course, similarities, but each had its own personality, its own processes and its own 
goals. This uniqueness played out in many ways and affected everything that happened. Issues 
included the division of resources, the push and pull between different efforts at each school, 
and staff turnover. What the teachers were willing or able to do or what might receive the most 
attention or gain the most status determined what was implemented. The educational reform 
witnessed at these sites could probably best be explained in terms of how schools change 
reform, not how reform changes schools (Tyack & Cuban, 1995). The schools changed reform 
to fit their own wants and needs, not the wants and needs of the reform project directors. 
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A critical element was the structure for change within the schools. This was a complex 
relationship and involved many aspects of the schools: the teachers specifically charged with the 
implementation, the other teachers, the liaison teacher and external requirements. The teachers 
were independent professionals, often worked in isolation and ultimately made their own 
decisions. It appeared that the teachers were most likely to implement the reform when it was 
their own choice. This does not mean that they could not be encouraged to change, but that the 
change took time and the teachers needed to develop ownership of the change. They had to feel 
that they were doing it because they wanted to. This fits with the constructivist philosophy in 
that learners need to construct their "own" understanding of a phenomenon before they truly 
accept it. 

Construction of understanding and the development of ownership take time. For an 
implementation to be successful it appears that it must be viewed as something that will 
continue and then actually be used for at least three years. This allows the teachers to try it out 
the first year, fix it and become more familiar with it the second year and then begin to make it 
their own in the third year. This model implies that the same teachers who attend training 
should also be responsible for implementing the changes and that they continue to be 
responsible for the change for at least the three years. It also implies that the group of teachers 
who are involved should be small enough so that each member of the team feels ownership and 
knows his or her input is needed. When pressures either inside or external to the school indicate 
that the curriculum may not last, teachers, probably wisely, do not invest the effort needed to 
develop ownership. Pressures can be positive or negative. For example, while this study was 
taking place there was heightened national interest in the National Science Education Standards, 
which the SS&C curriculum was designed to match. 

Coupled with the notion that it has to be the teachers' own idea, is the initial state of mind of the 
teachers. At the sites studied, teachers' reasons for becoming involved in the change varied 
from not really wanting to be involved at all, to thinking it was worth a try, to believing this was 
really the "right thing to do." These "prior conceptions" about the curriculum and training 
appeared to significantly impact the success of the implementation. Teachers who initially 
believed changing would be important were more likely to implement change and develop 
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ownership of it. Teachers who believed very strongly at either extreme either championed or 
sabotaged the change effort. Although reform efforts need champions, sometimes, these 
teachers were too pushy and ended up hindering the reform instead of helping it. Those in the 
middle are probably representative of most teachers involved in change. They want to do the 
"right thing" but are not sure what that is. They are usually concerned teachers who have been 
trying to do what they think is best for their students and so believe what they have been doing 
is good. This sets up a difficult belief changing situation. The teachers have to change what 
they think has been good in favor of something they are not sure of. To accomplish this, the 
teachers needed evidence that the change they were participating in was truly the "right thing." 
They needed support from their school and colleagues, and positive feedback from the students. 

The issue of support includes both material and emotional support. The notion of emotional 
support is complex. Teachers need support while experiencing change. They need to be led 
through the change carefully so that they will not feel defensive and reject or subvert it. They 
also need to be guided to truly understand the nature of the changes they are supposed to make. 
This takes a special type of support, a facilitator and guide, more than a banner waving, 
enthusiastic leader. It appears that at the sites studied, a guide who was not directly involved in 
the change was more effective, perhaps because this provided a perspective from someone who 
was not involved in the day to day challenges of teaching. It also appeared that these guides 
should be in a position of some power within the school structure. This provided the teachers 
with a conduit to the power structure and approval from someone in power. This sort of guide 
can also facilitate the obtaining of the recognition and materials teachers need to implement the 
curriculum. 

Material support is less complex but no less important. If a hands-on active science program is 
to be implemented, the teachers need materials. The materials need to be thoroughly tested to 
make sure they consistently work in the ways they should. Additionally teachers need to have 
enough supplies for all of the students to participate. Just having the necessary materials is not 
enough. The materials also have to be easily accessible to support the lessons. For example, 
one site put all of the materials and supplies necessary to support each set of lessons into boxes 
much like the elementary school kit science curricular programs. 
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Feedback from students is valuable to all, but especially to teachers who are unsure about the 
value of the change in the first place. The feedback is necessary to support the efforts involved 
in making the changes. At the sites studied, the information about student outcomes was both 
formal and informal. Students were seen as more interested in science or more engaged during 
class, as becoming better at designing experiments or using scientific reasoning, or anecdotal 
comments were supportive of the changes, etc. More formal information was provided by the 
sites or through the evaluation effort, such as enrollment data, increases on attitude toward 
science measures, success rates on out-of-class or standardized science tests, etc. If the student 
assessment results did not support the change or the assessments measured things not being 
covered by the current curriculum, teachers modified the curriculum to optimize student results 
on the assessment. 

Most educators anticipate that changes in classroom learning environments in terms of 
curriculum and instruction will be translated into more desirable student outcomes, i.e., if the 
teachers were indeed teaching better, the students should have better attitudes and perform 
better. Previous research has found that a positive learning environment and a more inquiry- 
oriented instructional technique can help increase student achievement (Fraser, 1986; Fraser et 
al., 1987; Shymansky et al., 1982). This was only partially the case in this study. Given the 
significant changes in the instruction documented in the first two years, there were surprisingly 
few increases in student outcomes although changes were seen at some sites in motivation, and 
lab skills performance improved at most sites. Since extent of implementation varied after the 
first two years, it is difficult to make overall statements. At the sites that continued, however, it 
appears there was a consistent gain in the lab skills performance. There was a very small 
overall negative effect on the science literacy multiple choice and open-ended tests. 

It is difficult to explain the lack of an effect given the strong changes in the learning 
environments. First it must be kept in mind that the achievement tests were not tied to the 
specific content covered in the SS&C curriculum. Tests were tied to the NRC standards and 
therefore were a very broad look at student achievement not a look that was designed to 
determine if a student learned what was taught. Nevertheless the tests were psychometrically 
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strong: the scales show strong factor structure and the reliabilities are high. The items were 
selected from existing and highly valid tests so the items themselves should be valid. 
Independent science education experts examined the tests and stated they matched the standards 
they were designed to measure. The tests show expected results in terms of higher scores for 
students who have higher grades in school, standard patterns for achievement for different 
ethnic subsets and gain in achievement for the sample overall from 9 th grade to 10 th grade. 

A possible explanation is that the learning environment has less effect on student performance 
than most would like to believe. Comparing the data in a slightly different way supports this 
notion. There were no differences in achievement in specific science areas between comparison 
and SS&C year students even when comparison students had studied a particular science area 
for the entire year while SS&C students had studied it for only part of a year. For example, in 
the 10 th grade comparison group sample almost all of the students took life science for the whole 
school year, yet the comparison group students did not score higher in life science than the 
SS&C students who only had life science part of the time. As another example, the SS&C 
students who had studied earth science did not perform better than the comparison group 
students who had not studied this area. 

Overall it appears that the reform effort was successful in causing changes in the way science 
was taught. The initial appearance and persistence of these changes, however, were dependent 
on the different elements present at the sites. Some sites and individual teachers gained and 
retained more from the reform effort than others. The institutionalization of change in a school 
was greatly affected by external pressures, the power structures in the school in relation to the 
reform, the availability of support and the desire for change. The data showed that if the same 
teachers continued to teach the reformed curriculum and worked on adapting it to match with 
their environments, their students' achievement was more likely to be affected. Perhaps, as 
teachers become more experienced with reform, they become more comfortable teaching it, 
teach it more effectively, and convey clearer expectations for student performance. 
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SS&C Model for Curricular Implementation 

The model for curricular implementation in Figure A was distilled from the SS&C experience. 

It is based on the notion that the intended curriculum is filtered and modified by a variety of 
different contextual factors. In the case of SS&C, the original curricular materials developed by 
NSTA were intended to be consistently implemented in all 13 sites across the country. In reality 
each site modified materials based upon the unique features of their school. 

The flow chart of curricular implementation in Figure A visually describes how the intended 
SS&C curriculum was first filtered through four inter-related factors: the community, the school 
and the students; the material and emotional support provided by the school; the teachers; and 
the liaison. Each site started with the materials developed by NSTA, but then considered how 
the materials could be used in their school with their own unique student population. The 
intended curriculum was also filtered through the support provided by the school and the 
resources available. The nature of the teachers, such as their content understanding, teaching 
philosophy, and openness to change, and of the liaison, such as his or her connections within the 
school power structure and facilitation skills, were also critical components of the curricular 
implementation. Ultimately, it was the teachers and the liaisons who made final decisions about 
the instruction and assessment that was used in the class. 

This resulted in the first enacted SS&C curriculum. This enacted curriculum was once again 
filtered through the same factors; however, this time around student outcome data and 
experiences with the curriculum were available. The teachers, parents and administrators had 
perceptions about the effectiveness of SS&C and these perceptions served as another filter for 
the curriculum. External pressures also impacted the SS&C curriculum. State mandated 
curriculum and assessment modified the implementation of SS&C at some sites. The result of 
this second filtering process was the implementation of the second enacted SS&C curriculum. 
Theoretically, the process could continue into the future. Each year the curriculum is modified 
and reconsidered based upon student outcomes, the local context, and external pressures. The 
result is a cyclical curriculum implementation process that reconsiders and implements a 
modified curriculum each year. 
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Figure A: SS&C Model for Curricular Implementation 
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Recommendations 

Recommendations for Effective Implementation Through Teacher Enhancement 



Efforts 



1 . Recognize that each school and teacher is unique and that consequently each 
implementation will also be unique. A project-wide curriculum will be modified by 
individual sites. 

2. Recognize that external pressures such as curricular and assessment requirements may 
significantly impact the new curricular implementation. 

3. Obtain district commitment to continue the implementation for at least three years 
with most of the same teachers. Effective change takes time. 

4. Obtain district commitment to provide the necessary facilities and materials as well as 
support a process for making the materials easily accessible. 

5. Involve teachers who either want to make a change or who are willing to try it and 
withhold judgment until later. Do not force the change on the teachers. 

6. Have small groups of the teachers who will be directly involved in the implementation 
attend training. 

7. Involve teachers in the development of the curriculum materials to help develop 
ownership and to help guarantee ease of implementation, but do not expect teachers to 
write the curriculum per se. The support of professional curriculum developers is 
needed. 

8. Provide personally supportive, local liaison teachers who have the knowledge 
necessary to be effective guides and who are part of the power structure. They can 
give the implementing teachers access to power and provide official support and 
sanction for their efforts. 
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9. Gather and provide evidence to the teachers and schools that the changes are 
producing valuable effects. 

10. Develop strong communication networks within and between sites, and between sites 
and the centralized project staff to increase teachers’ feelings of involvement and 
ownership in the project. The need for consistent and formalized avenues of 
communication increases as the number of implementation sites increases. 

Recommendations for Longitudinal Evaluations of Teacher Enhancement Projects 

1. Use the school or classroom as the unit of analysis with quantitative data and/or 
employ HLM techniques. 

2. Build rapport with the school and teachers to facilitate the data collection. Have a 
contact teacher in each school who can help evaluate to meet data collection needs. 

3. Carefully document everything that is done throughout the project and develop 
comprehensive filing systems that allow back-ups of all vital materials. 

4. Select evaluation team members who will be able to participate over the whole course 
of the study. 

5. Bring new people in and out of the evaluation team to provide new insights and 
energy. 

6. Have an external evaluation consultant to the evaluation effort who can review and 
refine your approaches. 

7. Produce publications or reports as the project progresses. These reports could be on 
the main topic or involve more research oriented special questions or interpretations 
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about the data being collected. 



8. Include sufficient funding for strong clerical help in formatting reports, tracking data, 
editing, and numerous other administrative tasks. 

9. Be vigilant in data analysis. Conduct routine crosschecks of data analysis 
procedures. Have more than one-person conduct analyses to serve as a crosscheck. 
Doing "side" studies can help to identify weaknesses in the analyses. 



V. INDIVIDUAL SITE DATA AND IMPRESSIONS 



This part of the report describes the data from each individual site in greater depth. For each 
site, the report first describes the context of the school qualitatively, including the nature ofthe 
community, school and its students; the nature of the teachers; the nature of the SS&C project 
liaison; the support provided by the school for the SS&C project; the external pressures on the 
school; and the progress of the implementation of the SS&C project. Then the report presents 
the quantitative data for each site. These data are presented in three sections: Learning 
Environment Measures, Student Outcomes: Attitude & Motivation, and Student Outcomes: 
Achievement. The charts for each site show trends over the six years of the study. For each 
site, identical calculations and analyses were used. These analyses are described in detail in 
Appendix B, Instrument Descriptions, and notable analyses are also described here. The tables 
in the Appendix present detailed, by item data in addition to the more aggregated data presented 
in the charts. 

• Unit of Analysis 

In all analyses, the student or teacher was used as the unit of analysis because the number of 
classes at the five schools was not large enough to allow using classes as the unit of analysis. 
The student and teacher sample sizes were small. For the instruments administered to the 
highest numbers of teachers and students, the teacher and student questionnaires, data were 
collected from a maximum of 6 and minimum of one teacher and a maximum of 397 and 
minimum of 10 students. 

• Statistical Significance 

For each state, changes were reported only when they were statistically significant at p<_ .05. 
Such changes were referred to as "statistically significant" or "significant" in the text. 

• Effect sizes 

Effect sizes for inclination, motivation, attitude, and all achievement measures were reported for 
each State on charts and in a table in the Appendix. Effect size (d) is a measure of the degree to 
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which two means differ in terms of the standard deviation of the parent population. In this study , 
the scores on all of the measures administered in the years after the reform was implemented 
(1996, 1997, 1998, 1999, 2000) were contrasted with the scores on the measures in the 
comparison year (1995). Because of the large differences between the years, the standard 
deviation was calculated for each pair of years separately. The measure of standard deviation 
used was the " square root of the pooled variance" for the two years being compared for each 
instrument. This procedure best controlled for statistical bias and error. The formula used was: 
d=Meanl -Mean2/square root of pooled variance within group- The effect size chart allows us to 
consider relative educational significance in addition to statistical significance. For the purposes 
of this report, an effect size of + 0.5 was considered to be educationally significant. 

• Science Literacy Test Analyses 

The science literacy open-ended test and multiple choice test data were analyzed using both z- 
tests and the Mann- Whitney U test because of low sample sizes. Results from the two statistical 
tests were similar, and z- scores were used here to determine significance and are reported in the 
tables in the Appendix. 

• Student and Teacher Questionnaire Factor Analyses 

The data relating to class activities were regrouped for the purposes of analysis on the student 
questionnaire and the teacher questionnaire. Factor analyses were conducted on both 
instruments using the data from the 1995 and 1996 years of the study because those years 
included data from all thirteen schools that were part of the original project. The sample sizes 
for those two years were therefore large enough to justify the use of factor analytic techniques. 

The table below shows the results of those factor analyses. Four factors emerged on the student 
questionnaire and two factors emerged on the teacher questionnaire. There was a great deal of 
correspondence between the factor analysis results on each instrument. On the inquiry scale, 
two of the four items making up the scale emerged on both the student and teacher 
questionnaires, Ask students to suggest hypotheses and Ask students to interpret data. The three 
items making up the traditional scale were the same on both the student and teacher 
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questionnaires, Read articles on science, Do an oral or written report, and Read other science 
materials. 



Scale 


Student Questionnaire 


Teacher Questionnaire 


Inquiry 


• Demonstration of a 
scientific principle 

• Ask students to suggest 
hypotheses 

• Ask students to interpret 
data 

• Relate previous work to 
current topic 


• Do experiments 

• Ask for reasons for the 
results of experiments 

• Ask students to suggest 
hypotheses 

• Ask students to interpret 
data 


Groups 


• Do experiments with 
other students 

• Work in groups 

• Share results from 
experiments 




Traditional 


• Read articles on science 

• Do oral or written report 

• Read other science 
materials 


• Read articles on science 

• Do oral or written report 

• Read other science 
materials 


Other 


• Watch films, slides or 
videos 

• Go outside for class 
instruction 





The frequency of the class activities included in the creation of the new groupings and the 
resulting new scale scores were reported on a five-point Likert scale ranging from 1= "Never" to 
5 = "Almost every day" for both the student and teacher questionnaires. Additional analyses 
were conducted on the student questionnaire exploring the percent of time spent in each kind of 
activity: inquiry, groups, traditional, or other. This percentage was calculated by adding 
together the scores for each scale in order to obtain a total score (inquiry + groups + traditional 
+ other). Then each scale score was divided by the total score and multiplied by 100 to obtain a 
percent of time spent in that activity (e.g. percent time spent in inquiry activities = (inquiry/total 
score) * 100). The results of these analyses are reported in Tables 1-4 in the Appendix and in 
each State's class activities charts. 
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VI. CALIFORNIA HIGH SCHOOL 



Detailed information about this site is contained in prior reports. What is presented here is a 
description of the implementation effort at the site in terms of the nature of the community, 
school, and the students, the teachers involved in the implementation, the liaison person, the 
emotional and material support provided by the school, external pressures and the progress of 
the implementation effort. 

Nature of the Community, School and Students 

During this study, California High School was a large, 9 th -12 th -grade high school located in an 
urban area in the State of California. The school suffered from problems of overcrowding, lack 
of resources, and lack of building maintenance. It had a large sprawling campus with several 
one-story concrete buildings. Because of an overcrowding problem, temporary classrooms were 
constructed near the football field. The buildings appeared run-down, dirty, and in need of 
regular maintenance. Some of the windows were broken, and graffiti could be found on many 
buildings. A chain- link fence surrounded the school, and students only entered the building 
through one door that was heavily monitored. Security at the school was tight, and police 
presence in the building was quite evident. Teachers also watched the halls closely during 
passing periods and were always on the alert for trouble. 

The student population at California High School was quite diverse. In fact, it was one of the 
few schools in this study where one could find approximately equal numbers of Caucasian, 
African American, Hispanic/Latino American and Asian American students all in the same 
class. The majority of the students at the school came from low income, working class families. 
Approximately one -third of the students in the school had a first language other than English. 
Most students were not college-bound and did not appear to put a lot of effort into their 
schoolwork. It was a daily struggle for teachers to engage the students in learning, keep them 
on task, and get them to turn in assignments. Many students failed classes or barely passed their 
classes. 
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Nature of the Teachers 

At California High School there were six class periods during the day, each lasting 50 minutes. 
There were approximately 20 classes of 9 th grade science with approximately 25 to 30 students 
per class. Over the years there were six or seven 9 th grade science teachers. Four of the teachers 
were at the school for more than 15 years, and these four teachers taught 9 th grade during the 
comparison year and taught SS&C science each of the following years. The other two or three 
9 th grade teaching slots changed from year to year. In each year of this project there was at least 
one new teacher teaching 9 th grade SS&C. These new teachers taught at the school for one or 
two years and then left. At times, the department had to fill one of the 9 th grade science 
openings with a long-term substitute because they were unable to hire a licensed science 
teacher. The short-term teacher and turnover issues appeared to affect both the comparison year 
and all subsequent SS&C years. 

There was a core group of four teachers who focused on implementing SS&C. Two of these 
teachers attended the summer workshops and training and led the SS&C implementation at this 
school. The core group of four teachers all appeared very committed to facing the challenge of 
working with the student body. They regularly arrived at school early and stayed late into the 
afternoon. They met as a team over the lunch hour and met regularly after school to discuss lab 
activities and modify the new SS&C curriculum. During the implementation of SS&C this core 
group of 9 th grade teachers also worked long hours gathering materials, modifying lab activities 
to fit their equipment, and discussing changes after trying out activities. 

Nature of Liaison 

The liaison at this school was the science department chair during the first three years of the 
project. After three years another veteran teacher took over department chair responsibilities. 
The liaison was one of two teachers who attended the national summer workshops and training 
sessions, and led the SS&C implementation at California High School. During each year of the 
project he also taught several sections of 9 th grade SS&C. He led the pilot testing and revision 
of SS&C activities, and was a leader in helping the new teachers and the long-term substitutes 
learn how to implement SS&C. The other teachers in the department appeared to respect his 
expertise and followed his advice. He was also instrumental in obtaining additional resources 
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from the principal to organize the equipment for each SS&C activity so that in future years they 
could easily access the materials they needed for each activity. 

Support Provided by the School 

The principal at California High School was very supportive of SS&C. During an interview, he 
said that the science department was one of the best departments in the school, and he was quite 
impressed by their commitment and desire to make their classes better. He was disappointed in 

till 

the low science test scores and concerned about the large percentage of students that failed 9 
grade science, but he was encouraged that the department was at least attempting to improve the 
situation. Money and space were both at a premium in the school. Despite these constraints the 
principal provided money to help purchase additional equipment and storage cabinets so the 
science department could organize the materials. In addition, the administration added running 
water to several science classrooms to help teachers implement SS&C. 

External Pressures 

In the third year of SS&C the State of California required all schools to administer a 40 question 
multiple choice physical science test at the end of 9 th grade. This placed pressure on the science 
department to reorganize the subject matter emphasis in the 9 grade. SS&C was originally 
designed to include an equal balance of physics, chemistry, biology, and earth science. Because 
of the new State physical science test, the department decided to shift the 9 th grade course focus 
to physical science and cover biology and earth science in the 10 grade. 

In addition, the new State mandated physical science test pressured the department to alter 
assessment methods. When SS&C began, the 9 th grade teachers shifted to predominately essay, 
problem solving, and performance-based assessments. However, when the state began the 
multiple choice testing, the department decided to revert back to more multiple choice testing in 
order to prepare students to answer these types of questions on the end of the year test. They 
did not completely revert back to all multiple choice tests, but they definitely shifted the 
emphasis back to traditional testing. 
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Progress of the Implementation 

In the first year of the project California High School converted all of their traditional 9 th grade 
physical science classes to SS&C. During the first two years of the project a core group of 
teachers made a good faith effort to use an inquiry^oriented sequence as prescribed by SS&C. 
During the third year of SS&C California High School continued to implement SS&C as 
intended; however, they slowly began to replace and modify some of the SS&C activities that 
they did not think worked well. They still implemented approximately half of the priority 
SS&C activities; however, they used many of their own activities too. Despite the alterations in 
the activities, the philosophy of SS&C remained intact. The new activities used by the teachers 
used an SS&C-like sequence, meaning that they were more inquiry oriented in nature. In 
addition, it was during the third year of the project that California High School shifted the 
physical science activities to the 9 th grade in response to the new end-of-the-year physical 
science test. To date, the science department continues to use this format for 9 grade science. 

Based on the teachers' reports of what topics were covered each year, with the introduction of 
SS&C there was a decrease in the amount of earth science instruction, and an increase in 
physics and biology. (See Table 27 in the Appendix.) 

California High School Learning Environment Measures 

The learning environment was assessed in four different ways. The students and teachers 
completed a questionnaire that asked about what activities occurred in the class. These 
activities were grouped into factors. On the student questionnaire there were four factors: 
inquiry, group work, traditional, and other. On the teacher questionnaire there were two factors: 
inquiry and traditional. The introduction to the individual site sections describes these factors 
and their analyses in detail. The third measure was observations of the classes by external 
observers. These observations allowed the class activities to be categorized into student 
centered, teacher centered, group, administrative and other. The fourth measure was a learning 
environment inventory with several scales related to the classroom psychosocial environment, 
which was completed by the students. 
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The learning environment at California High School changed significantly during the course of 
the SS&C project. When forced into percent of total class time categories, the student 
questionnaire data showed a significant increase in the percentage of time students worked in 
group activities during all five years of the study. In 1996 and 1997 there was a slight 
significant decrease in the percentage of time teachers used inquiry-based activities; however, 
students reported a significant, slightly more frequent use of inquiry-based activities. Inquiry 
activities occurred approximately once a week during the comparison year and somewhere 
between once a week and several times a week during the SS&C years. The percentage of time 
spent on traditional lecture-based activities stayed constant during the project in all but one year. 
During the majority of the years of the project, students reported a significant increase in 
specific inquiry teaching techniques such as demonstrating scientific principles, relating 
previous work to the current topic, and asking students to interpret data. They also reported an 
increase in specific group activities such as doing experiments with other students, working in 
groups, and sharing results of experiments. 



Figure 12 

Student Questionnaire - California 
Percent of Time Spent by Class Activities 
Student Means 

Inquiry """Group Traditional Other 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 
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The teacher questionnaire data were not forced into percentages like the student data. Only the 
frequencies are presented below. There was a significant increase in inquiry teaching during the 
third and fourth years of SS&C. Traditional teaching increased significantly during the second 
year only, but otherwise remained constant throughout the project. 



Figure 13 

Teacher Questionnaire - California 
Frequency of Class Activities 
Teacher Means 

= Inquiry - - - Traditional 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 

During the last three years of the project, questions were added to the teacher questionnaire. 
The teachers were asked to rate the percent of time they spent in whole class activities, group 
activities and individual activities. After doing these ratings the teachers then described what 
happened in each category. The teacher reports on the percent time spent on group versus 
whole class activities showed that group activities were more common during the last three 
years of SS&C. Of the group instruction activities, hands-on activities seemed to dominate. In 
terms of whole class activities, class discussions were most common. (See Tables 34-36 in the 
Appendix.) 
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The class observations revealed that during the comparison year there were approximately equal 
amounts of teacher-centered and student- centered teaching. During the first year of SS&C there 
was a significant increase in teacher- centered activities (approximately 40% of class time), 
followed by a decrease back to comparison year levels in the second year (30% of class time). 

In the third year of SS&C there was a significant increase in student-centered instruction (50% 
of class time). The fourth and fifth years there was a move back towards more teacher-centered 
and more teacher- and student-centered instruction (30% each year). Overall, the learning 
environment measures appeared to parallel the implementation of SS&C at California High 
School. SS&C produced more group activities and more inquiry-based teaching; however, the 
use of inquiry techniques was difficult to sustain. The changes in instruction appeared to peak 
in the third year of the project and began to revert to comparison year levels by the end of five 
years. 



Figure 14 

Classroom Observations • California 
Percent Time Spent by Teaching Style 
■ ■ ■ Teacher-Centered Administrative 

Student- &Teacher-Centered Other activities 

“ ■ ” Student-Centered 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 



As a second look at the classrooms, the observation forms asked the observer to rate the level of 
student involvement and type of instruction. The ratings of student involvement showed no 
significant changes over the project. There was a significant increase towards more inquiry- 
oriented instruction during the last three years of SS&C. 
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The learning environment inventory scales indicated the SS&C curriculum was viewed as 
significantly less difficult during all five years of the project. The sequencing and 
experimenting scales indicated that SS&C used a significantly more inquiry- oriented sequence 
during the first four years of the project, and significantly more experimenting during the first 
three years of the project. 



Figure 15 

Learning Environment Inventory - California 
Student Means 

“ ” ” Difficulty Involvement 

Relevance Sequencing 

“ ■ " Critical Voice — Experimenting 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 



There was a relatively significant amount of variability in scores from year to year. This 
seemed to correspond to the changes in teachers from year to year and to the difficulty this 
created in terms of consistent implementation. 



California High School Student Outcome Measures 

The student outcome information is grouped in two ways. The inclination of students to 
continue studying science and student motivation and attitude toward science is one group, and 
student achievement on the science literacy multiple choice items, science literacy open-ended 
items, lab skills performance test, and written full investigations is the second. 

Attitude & Motivation Measures 

There were only minor changes in attitudes at California High School. In 1997, students’ 
inclination to take science in 10 th grade significantly increased slightly. In 1997, 1998, and 
2000 there was a significant decrease in the percentage of students who stated that the activities 
in science class made them want to take more science and that science class is fun. There was 
also a significant decrease in 1997 in the percentage of students who found class motivating, 
and a significant decrease in 2000 in the percentage of students who found class interesting. In 
2000, there was a slight significant increase in the percentage of students that stated the things 
they learn in science class relate to the real world. 



Figure 16 

Student Questionnaire - California 
Inclination to Continue Studying Science 
Percentage of Students who Responded "YES" 

■ ■ ■ Take science in 10th grade “ ■ ■ Take science in 12th grade 

Take science in 11th grade Take science in college 
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Figure 17 

Student Questionnaire - California 
Science Motivation 

■ ■ - Career in science? " • ■ Is your science class motivating? 

Activities lead to more science? Had a "totally awesome" experience in class? 
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Student Questionnaire - California 




Attitudes Toward Science 


■ ■ - Science is useful 


Science in necessary to get a good job 




— Science class is interesting 


— ■ ~ Science will be useful in the future — Science class is fun 
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Achievement Measures 

Several achievement measures at California High School revealed significant increases over the 
comparison group. The science literacy multiple choice scores were significantly higher than 
the comparison group in 1997, 1998, and 2000. The hands-on full investigation scores were 
significantly higher in 1997, and the open-ended scores were significantly higher in 1998. 
However, the written full investigation scores were significantly lower than the comparison 
group in 1996, 1997, 1998, and 1999. The lab skills performance test scores remained steady 
throughout the project. The positive changes on students' science literacy test scores appeared 
to correspond to the slow transition into SS&C methods that were observed in the classroom. 
The first year of SS&C was difficult, but after the teachers worked at the problem and got better 
organized, positive changes in achievement were observed on three of the five measures. The 
effect size chart revealed positive educationally notable changes for the hands-on full 
investigation (1996 and 1997) and the science literacy open-ended questions (1998). Negative 
educationally significant changes occurred for the written full investigation in all years and for 
the lab skills performance test (1997). 



Figure 19 

Achievement Scores - California 
Student Means 

Percent Correct on Achievement Tests 

- - - Lab Skills Performance Test Written Full Investigation 

Science Literacy Multiple Choice Items Science Literacy Open-Ended Questions 

— * - Hands-On Full Investigation 
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Figure 20 
California 

Achievement Effect Sizes 

Lab Skills Performance Test Written Full Investigation 

Science Literacy Multiple Choice Items Science Literacy Open-Ended Questions 

“ ■ “ Hands-On Full Investigation 
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Summary of California High School 

In summary, the implementation ofSS&C at California high school fits nicely with the model of 
curriculum implementation. SS&C was influenced by a variety of contextual and State factors. 
The intended SS&C curriculum was modified in light of the nature of the students. California 
High School had many students with limited English proficiency, and to accommodate these 
students the SS&C materials were modified. Fewer activities were included because the 
students worked at a slower pace and simpler readings were included. In addition, SS&C was 
modified because of the lack of resources and lab facilities. Like many urban schools they 
lacked expensive equipment and modified by using everyday materials for activities. The first 
enacted curriculum was essentially a simplified version of the intended curriculum. External 
factors played a role in subsequent revisions of SS&C. When the State of California mandated a 
new end -of- the- year physical science test it forced the teachers to alter the curriculum by 
shifting more physical science activities to the 9 th grade. It also forced a shift back to more 
traditional assessment to help students prepare for the multiple choice test mandated by the 
State. The enacted curriculum ultimately stabilized and at of the end of the project was still 
being used by the 9 th grade science teachers at California High School. 
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VII. IOWA HIGH SCHOOL 



Detailed information about this site is contained in prior reports. What will be presented here is 
a description of the implementation effort at the site in terms of the nature of the schools and the 
students, the teachers involved in the implementation, the liaison person, the emotional and 
material support provided by the school, external pressures and the progress of the 
implementation effort. 

Nature of the Community, School and Students 

The high school was located in the suburbs of a medium- sized Iowa city. During the study, the 
surrounding suburban area consisted of moderate sized single family homes. There were multi- 
unit apartments within short distances from the school. New construction of single and multi- 
units was a common sight in the area. Older, established residential areas were located further 
away from the school, and in the city proper. The community ranged economically from low to 
upper middle class. The student population was almost entirely Caucasian with few African 
American, Asian American and Hispanic/Latino American students. The school had a high 
number of students, above 90%, who continued on to post- secondary school opportunities. The 
community and the population of the school were relatively stable over the course of the study. 
There were several large industries that had been in the community for many years, including 
John Deere Tractor, which were very active in providing technical and engineering jobs as well 
as support programs for the schools within the district. 

The school was a 30-year-old two story brick/cement block building, with a science and math 
brick/cement block addition that was 1 0-years-old. The school contained an L-shaped floor 
plan. The building was very clean and neat with large plants greeting the students as they 
entered the school. Both inside and outside, the building was pleasant to look at and was well 
maintained. The school was located on a large tract of land, which the horticulture class used as 
an outdoor classroom. The school site inc luded a state of the art track and field, and complete 
athletic facilities, including a swimming pool. The science classrooms were grouped together in 
the new addition, located on both the first and second floors. Class sizes over the years ranged 
from 18-25. Classes were offered on a 45-minute period schedule. The students were generally 
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polite and behavior problems were few. The attendance rate was above 95%, and students 
seemed to be involved in and generally accepting of their school programs. 

Nature of the Teachers 

There were seven science teachers in the school each of the data collection years with two 
teaching 9 th grade. All of the science teachers were dedicated to making the science experience 
the best they could for their students. The two teachers who attended the initial training taught 
the SS&C course for the first two years, and then one new teacher and the next year another, 
were hired that took over the 9 th grade teaching. The prior teachers mentored these new 
teachers. The teachers felt they were doing a good job prior to the implementation of the SS&C 
project, but also were willing to try new things if it would help the students. 

Nature of the Liaison 

There were two different liaison teachers during the period of the study. Both individuals were 
also in turn the heads of the science department. The second liaison assumed the duties after the 
original liaison retired. The liaison teachers did not teach SS&C but they both were very 
supportive of the teachers who were actively involved with the project. Both individuals 
answered questions and provided adequate support for supplies and for paid release time to 

th 

work on the curriculum. Both liaisons believed that the course was really in the hands of the 9 
grade teachers teaching the SS&C course. 

Support Provided by the School 

There were two principals during the period of this study. Based on the interviews, both 
principals appeared to be concerned about providing students with a quality education, one that 
prepared them for the diverse opportunities that they would face after leaving high school. Both 
principals also appeared to be concerned about enhancing the reputation of the school. The 
school held high standards for academic achievement and was a proud member of the local 
community. 

The laboratories were reasonably well equipped with most of the major equipment being 
somewhat old. The teachers purchased consumable supplies and materials as needed for the 
science program, using science budget funds. Major pieces of equipment were purchased on a 
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need basis. The school provided summer salary for the teachers for several years in order to 
work on consolidating and improving the 9 th grade science curriculum. 

External Pressures 

There were no significant external pressures on the school. The Statewide assessments 
remained the same, and no serious financial or enrollment changes occurred. 

Progress of the Implementation 

Teachers who were trained during the summer used the curriculum the first year. These 
teachers taught SS&C the next year, and then one new teacher was hired and then a second 
teacher was hired for the 9 th grade. The two new teachers are now in charge of the 9 th grade 
course. The originally trained teachers played a key mentoring role. The curriculum contained 
much of the original SS&C material but was consolidated, added to and rearranged to better fit 
the school and students. The last year of the study the school used an integrated science 
textbook as well as the SS&C materials. The administration, as well as concerned parents, 
supported the addition of the textbook. The integrated science course is required for all ninth 
graders except a small group of honors students. 

Based on the teachers' reports of what topics were covered each year, the first year of SS&C 
showed an increase in the time spent teaching physics and a decrease in the time spent teaching 
biology. After that first year the amount of time spent teaching the four science areas of 
physics, chemistry, earth science and biology were somewhat balanced around ten weeks each, 
with earth science showing the most fluctuation. (See Table 53 in the Appendix.) 

Iowa High School Learning Environment Measures 

The learning environment was assessed in four different ways. The students and teachers 
completed a questionnaire that asked about what activities occurred in the class. These 
activities were grouped into factors. On the student questionnaire there were four factors: 
inquiry, group work, traditional and other. On the teacher questionnaire there were two factors: 
inquiry and traditional. The introduction to the individual site sections describes these factors 
and their analyses in detail. The third measure was observations of the classes by external 
observers. These observations allowed the class activities to be categorized into student- 
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centered, teacher- centered, group, administrative and other. The fourth measure was a learning 
environment inventory with several scales related to the classroom psychosocial environment, 
which was completed by the students. 

When forced into percent of total class time categories, the student questionnaire data showed a 
significant increase in group activities that continued throughout all the years of the project. 
This increase was accompanied by a significant decrease in independent inquiry activities. 
Overall, the students rated the frequency of inquiry activities as about once a week to several 
times a week and the frequency of group activities as several times a week. The frequency of 
traditional activities was rated as less then once a week. 



Figure 21 

Student Questionnaire - Iowa 
Percent of Time Spent by Class Activities 
Student Means 

Inquiry - - - Group Traditional Other 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C31998 SS&C41999 SS&C 5 2000 
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The teacher questionnaire data were not forced into percentages like the student data. Only the 
frequencies are presented below. There was one significant change, an increase in traditional 
activity the first year but then returned to original levels. The 1997 data are missing. The 
teachers rated the traditional activities as occurring slightly less than once a week and inquiry 
activities as increasing over the course of the study, although not significantly, from about once 
a week to more than several times a week. 



Figure 22 

Teacher Questionnaire - Iowa 
Frequency of Class Activities 
Teacher Means 

= Inquiry - - - Traditional 




Comparison 1995SS&C 1 1996 SS&C21997 SS&C 3 1998 SS8.C4 1999 SS8.C 5 2000 

During the last three years of the project, questions were added to the teacher questionnaire. 

The teachers were also asked to rate the percent of time they spent in whole class activities, 
group activities, and individual activities. After doing these ratings, the teachers then described 
what happened within each category. These data showed that in 1999 there was an increase in 
individual activities and a decrease in group activities, but that these changed back to original 
levels in 2000. Individual hands-on activities showed increases over the three years. (See 
Tables 60-62 in the Appendix.) 



The classroom observation data showed a significant increase in the time spent displaying 
student -centered teaching during the first SS&C year, which was maintained through all years. 
The time spent displaying student- and teacher- centered teaching dropped significantly during 
the first SS&C year and remained lower than the comparison year all years. The percent of time 
spent in student-centered teaching increased from about 10% to 40% the first year and then 
increased again to about 60% the last year. 



Figure 23 

Classroom Observations-lowa 
Percent Time Spent by Teaching Style 
■ “ “ Teacher-Centered Administrative 

Student- & Teacher-Centered Other activities 

— ■ - Student-Centered 
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As a second look at the classrooms, the observation forms asked the observer to rate the level of 
student involvement and the type of instruction. These ratings showed no significant changes 
but there was a trend toward more student involvement and more mixed instruction. 
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The learning environment inventory showed the involvement, relevance and critical voice scales 
significantly rose together after the second year of SS&C and remained high for the rest of the 
years except involvement the last year. The scale scores for these three items spanned from 
sometimes to often. Sequencing increased significantly the first year but then returned to and 
remained at initial levels. Difficulty decreased significantly the third and fourth years and then 
returned to comparison year levels the final year. In all years the experimenting scale was 
significantly higher than the comparison year. 

Figure 24 

Learning Environment Inventory - Iowa 
Student Means 

■ ■ ■ Difficulty Involvement 

■■■■■ Relevance Sequencing 

— ■ ■ Critical Voice — ■■■ Experimenting 
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Overall the changes in classroom learning environment documented by the various measures 
indicated that the classrooms moved toward more inquiry-based student-centered learning that 
was perceived as being more relevant, more involving and as allowing more experimentation 
These changes reflected the implementation of the reform, which began well and continued to 
be maintained throughout the course of the study. 
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Iowa High School Student Outcome Measures 

The student outcome information can be grouped in two ways. The inclination of students to 
continue studying science and student motivation and attitude toward science is one group, and 
student achievement on the lab skills performance test, science literacy multiple choice items, 
full investigations, and science literacy open-ended items is the second. 

Attitude & Motivation Measures 

Little change was seen in the inclination of students to take science courses in 10 th and 1 1 th 
grade. This was probably due to the school's graduation requirements. There was, however, a 
significant rise in inclination to take 12 th grade science that remained all years of the study. The 
“totally awesome experience” item was significantly higher than the comparison year the first 
year of SS&C and throughout the rest of the study remained high. The “activities make you 
want to take more science,” "science class is motivating," and "had a ‘totally awesome’ 
experience" items were significantly higher than the comparison year in 1997 and remained 
there. The attitude data generally showed significant increases starting the second year of SS&C 
implementation. Most of these increases remained for the course of the study. 



Figure 25 

Student Questionnaire - Iowa 
Inclination to Continue Studying Science 
Percentage of Students who Responded "YES" 

■ ■ Take science in 10th grade ™ ■ ■ Take science in 12th grade 

Take science in 11th grade Take science in college 
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Figure 26 

Student Questionnaire - Iowa 
Science Motivation 

Percentage of Students who Responded "YES" 

“ “ “ Career in science? “ ■ “ Is your science class motivating? 

Activities lead to more science? Had a "totally awesome" experience in clas 
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Figure 27 

Student Questionnaire - Iowa 
Attitudes Toward Science 

" “ “ Science is useful Science in necessary to get a good j 

Science is relevant Science class is interesting 

“ ■ “ Science will be useful in the future Science class is fun 
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Achievement Measures 

Some significant changes were seen in the achievement data. Students scored significantly 
lower on the science literacy multiple choice test all years of the study. They scored 
significantly higher on the lab skills performance test only the last year of the study but had 
been showing consistent improvement in their scores since the second year of SS&C. The 
scores on the science literacy open-ended items showed significant decreases from the 
comparison year the first, fourth, and fifth years of SS&C. The other scores varied but did not 
show significant changes. The effect size chart revealed positive educationally notable changes 
for the hands-on full investigation. Negative educationally notable changes occurred for the 
science literacy multiple choice and open-ended questions. 



Figure 28 

Achievement Scores-lowa 
Student Means 

Percent Correct on Achievement Tests 
" " " Lab Skills Performance Test Written Full Investigation 

Science Literacy Multiple Choice Items “ ““ Science Literacy Open-Ended Questions 

“ ■ " Hands-On Full Investigation 
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Figure 29 
Iowa 

Achievement Effect Sizes 

" " " Lab Skills Performance Test Written Full Investigation 

Science Literacy Multiple Choice Items ““ Science Literacy Open-Ended Questions 

— ■ “ Hands-On Full Investigation 
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Summary of Iowa High School 

In summary the Iowa High School implementation was complete. Considering their experience 
in light of the proposed model for teacher enhancement highlights several crucial components. 
The school was well prepared to implement the curriculum. It was well supplied with the 
equipment necessary to support the curriculum, the teachers were well qualified, and the 
administration, i.e., the principal and the department chair- liaison teacher, was supportive. The 
two teachers were somewhat skeptical about the value of the new curriculum but were willing to 
see what it had to offer. The first enacted curriculum was very close to the intended curriculum 
with slightly less student-centeredness than the curriculum called for. The teachers felt the 
curriculum had promise and modified it for their use the second year, bringing in more of their 
own activities and modifying the ones they felt did not work as well as they might. The classes 
became more and more student-centered over the years. The teachers were given extra time 
each summer to work on the curriculum. This pattern of incremental improvement continued 
throughout the course of the evaluation project. After several rounds of improvement the initial 
teachers passed the course on to new teachers one at a time, and these new teachers were 
engaged in the incremental improvement process. 



O 

ERiC 



72 



77 



VIII. MONTANA JUNIOR HIGH SCHOOL 



Detailed information about this site is contained in prior reports. What will be presented here is 
a description of the implementation effort at the site in terms of the nature of the community, 
school, and the students, the teachers involved in the implementation, the liaison person, the 
emotional and material support provided by the school, external pressures and the progress of 
the implementation effort. 

Nature of the Community, School and Students 

During this study, Montana Junior High School was an 8 th and 9 th grade junior high school 
located in a small city in the Rocky Mountains of northwestern Montana. The school itself was 
a traditional two-story brick building. It was clean and well maintained, and all indications were 
that the school was well funded and well equipped. The atmosphere of the building was very 
upbeat, and the staff and students were quite friendly and welcoming. The school in many ways 
had a typical junior high feel; it was very active and social. At Montana Jr. High there were 
eight class periods during the day, each lasting approximately 50 minutes. All of the science 
classrooms were located in one area of the building. The school had over 500 9 th graders, who 
were predominately middle-class and Caucasian. The majority of students were college bound 
and appeared motivated about learning and doing well in school. 

Nature of the Teachers 

There were two 9 th grade science teachers who were involved in the SS&C program from the 
very beginning. Only one of these two teachers attended summer SS&C workshops. They were 
both veterans that had been in the field for over 15 years. They worked hard together at 
planning the SS&C lessons and gathering all the equipment necessary to implement the SS&C 
activities. These two teachers, along with one other teacher at the school, taught all of the 
sections of SS&C for the first three years of the project. At that time, one of the lead teachers 
retired. In addition, the other lead teacher, who also served as department chair, decided to 
rotate new teachers into SS&C. As a result, in the fourthyear of the project all of the teachers 
were new to SS&C. The lead teachers organized the materials so well that transitioning in new 
teachers was quite smooth. They scheduled classes so that most SS&C classes met at a different 
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time, and as a result they could share materials. The materials were placed on a cart. One 
teacher used them and then passed them on to the next teacher. This also allowed the first 
teacher to pass on tips to the other teachers and to discuss what went well and what needed 
modification. 

Nature of the Liaison 

The liaison at this site was the high school science department chair and chemistry teacher in the 
same district as the junior high. He did not teach SS&C, but served as an advisor to the 
implementation and development of activities. He was extremely supportive of SS&C and 
served as one of the developers of chemistry activities for the national project. During the first 
year of the project he went to the junior high school several afternoons a week to meet with the 
two lead teachers about implementing SS&C and refining activities. In subsequent years he 
visited the junior high at least monthly and as needed by the teachers. Through his position as 
department chair he was also able to provide money for new equipment as needed by the 
teachers, and a vision of how the 9 th grade SS&C activities fit into the district science scope and 
sequence curriculum. 

Support Provided by the School 

The principal at Montana Jr. High School was very supportive of SS&C and believed that the 
teachers did an excellent job of implementing the curriculum. Based on the increased student 
enrollment in SS&C the principal was quite pleased with how the course was received and 
believed the program succeeded and served students very well. The principal also thought the 
teachers did an outstanding job of implementing SS&C. 

External Pressures 

The primary external pressure on the SS&C program was the tension between teachers and the 
principal regarding the number of science course offerings. The principal felt strongly about 
consolidating science course offerings down to one or two options in order to make it easier to 
schedule classes. At the same time, the teachers felt strongly about offering many different 
options to the students including biology, SS&C, basic SS&C, and honors earth science. 
Ultimately, enrollment in SS&C was left up to students and parents. At first, many parents were 
concerned about converting all courses to SS&C. SS&C classes were therefore started as an 
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optional alternative to taking biology or earth science. Over the years, interest and enrollment in 
SS&C increased as more and more students and parents saw the impact on students. At the end 
of the project, the principal still desired a complete conversion to SS&C, while the teachers 
wanted to allow students and parents to choose from different course options. 

Progress of the Implementation 

In the first year Montana Junior High offered seven sections of SS&C, two were low-level 
SS&C, and five were regular SS&C. Both tracks did essentially the same activities; however, 
the low-level classes did not go into as much depth and took extra time to discuss each activity. 
By the end of the five year project, the enrollment in SS&C had increased to ten sections of 
SS&C. Throughout the project the science department continued to offer biology and honors 
earth science for their advanced students who did not want to take SS&C. 

All of the 9 th grade teachers were required to teach SS&C at some point. The department chair 
had everyone teach SS&C on a rotating basis so that everyone had a common experience and 
ownership over the curriculum. Montana Junior High School used the SS&C curriculum as 
intended throughout the entire project. They only made minor modifications to help the 
activities fit their equipment. In addition they used the original 9 th grade sequence provided by 
SS&C. The only aspect of the curriculum that they did not use was the readings. Prior to SS&C 
they used labs and believed in hands-on science, but with SS&C they gave students much more 
control and input into doing science. In the past, labs were traditional cookbook- style labs with 
all the procedures dictated to students. The more inquiry- oriented SS&C methods opened their 
eyes to how much more students could do. At the end of the five year project, SS&C was still 
being implemented as intended with all indications that the science department would continue 
offering SS&C well into the future. 

Based on teachers' reports of what topics were covered each year, the first year of SS&C 
included more physics, chemistry and biology, and less earth science. Prior to SS&C most 
students took earth science in 9 th grade, and the change to SS&C resulted in balanced instruction 
in all four areas. (See Table 79 in the Appendix.) 
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Montana Junior High School Learning Environment Measures 

The learning environment was assessed in four different ways. The students and teachers 

completed a questionnaire that asked about what activities occurred in the class. These 
activities were grouped into factors. On the student questionnaire there were four factors: 
inquiry, group work, traditional, and other. On the teacher questionnaire there were two factors: 
inquiry and traditional. The introduction to the individual site sections describes these factors 
and their analyses in detail. The third measure was observations of the classes by external 
observers. These observations allowed the class activities to be categorized into student- 
centered, teacher- centered, group, administrative and other. The fourth measure was a learning 
environment inventory with several scales related to the classroom psychosocial environment, 
which was completed by the students. 

The student, teacher and observation data all supported the conclusion that the learning 
environment at Montana Junior High School was significantly changed throughout the entire 
SS&C project. When forced into percent of total class time categories, the student questionnaire 
results showed there was a significant increase in the percentage of time teachers engaged in 
both inquiry and group activities, and a significant decrease in the percentage of time teachers 
used traditional activities. On specific questions about class activities the students indicated that 
from the comparison year to all SS&C years there was a significant increase in the frequency of 
having students do experiments with other students, share the results of experiments, suggest 
hypotheses (except in 1 997), and work in groups. 
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Figure 30 

Student Questionnaire - Montana 
Percent of Time Spent by Class Activities 
Student Means 

Inquiry - - - Goup Traditional Other 
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The teacher questionnaire data were not forced into percentages like the student data. Only the 
frequencies are presented below. There was a significant increase in the use of inquiry during 
all but one of the SS&C years. 



Figure 31 

Teacher Questionnaire - Montana 
Frequency of Class Activities 
Teacher Means 

= inquiry - - - Traditional 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 

During the last three years of the project, questions were added to the teacher questionnaire. 
The teachers were asked to rate the percent of time they spent in whole class activities, group 
activities and individual activities. After doing these ratings the teachers then described what 
happened in each category. Teachers' views of the percent of time on group or whole class 
activities showed that during all years teachers used group activities more than whole class 
activities. Hands-on activities dominated the group instruction, while class discussion 
dominated the whole class instruction. (See Tables 86-88 in the Appendix.) 
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The class observations also showed that the learning environment changed significantly with the 
use of SS&C. During the comparison year observations of classes indicated that teacher- 
centered activities dominated instruction (approximately 70% of class time). During the first 
year of SS&C teacher-centered activities dropped down to 15% of the class time while student- 
centered activities increased to over 50% of the class time. Student -centered instruction 
dominated the instruction over all years of the project. 



Figure 32 

Classroom Observations - Montana 
Percent Time Spent by Teaching Style 

- - - Teacher-Centered Administrative 

Student- & Teacher-Centered Other activities 

— ■ - Student-Centered 
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As a second look at the classrooms, the observation forms asked the observer to rate the level of 
student involvement and type of instruction. The ratings of student involvement showed no 
significant changes over the project; however, there was a significant increase towards more 
inquiry- oriented instruction during all years of SS&C. 





The learning environment inventory results also showed a significant change in the learning 
environment. There was significant increase in the use of an inquiry sequence during all SS&C 
years, a significant increase in students’ critical voice and the extent to which students 
experimented in two of the five years. There was a significant decrease in students’ perception 
of the difficulty of classes in all SS&C years except the last one. 



Figure 33 

Learning Environment Inventory - Montana 
Student Means 

■ ■ ■ Difficulty Involvement 
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The changes in the learning environment directly corresponded to the implementation of SS&C 
at Montana Junior High School. The two lead teachers diligently implemented SS&C and 
created a kit-based system of SS&C activities where new teachers could take o\er and 
implement SS&C with the support of an experienced teacher. The system appeared to have 
resulted in a consistent implementation of SS&C and consistent changes in the learning 
environment during each of the years of the project. 



Montana Junior High School Student Outcome Measures 

The student outcome information is grouped in two ways. The inclination of students to 
continue studying science and student motivation and attitude toward science is one group and 
student achievement on the science literacy multiple choice items, science literacy open-ended 
items, lab skills performance test, and written full investigations is the second. 

Attitude & Motivation Measures 

Despite the dramatic changes in the learning environment at Montana Junior High School, there 
were slight or no changes in students’ attitudes and motivation. The inclination to study science 
in the future appeared to remain constant. Only very minor changes were noted in students’ 
attitudes toward science and motivation. In the first year only there was an increase in students' 
perceptions that their science class was motivating, and in two of the SS&C years there was an 
increase in having a totally awesome science experience. No changes were found in students 
indicating that they intend to study more science in 10 th , 1 1 th , 12 th grade or in college. It 
appeared that even though the teachers used a more inquiry- oriented approach to teaching 
science, it did not have a significant impact on student attitude or motivation. Scores on these 
measures were positive during the comparison year and remained positive during the SS&C 



program. 



Figure 34 

Student Questionnaire - Montana 
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Student Questionnaire - Montana 
Science Motivation 

■ ■ ■ Career in science? “ ■ " Is your science class motivating? 

■ Activities lead to more science? Had a "totally awesome" experience in class? 
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Student Questionnaire - Montana 




Attitudes Toward Science 


■ ■ ■ Science is useful 


Science in necessary to get a good job 


Science is relevant 


Science class is interesting 


“ ■ “ Science will be useful in the future — science class is fun 
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Achievement Measures 

No significant increases were found on the science literacy multiple choice items. For two 
years, significant decreases were found on the open-ended science literacy items. The lab skills 
performance test scores during four of the five years of the SS&C project were significantly 
higher than the comparison group. Differences were also found on the hands-on full 
investigation test. In three of the five years the SS&C group scored significantly higher on the 
hands-on full investigation tests than the comparison group. SS&C students also scored 
significantly higher on the written full investigation test during only one year. The differences 
in hands-on test scores appeared to be due to the more inquiry- oriented science teaching 
methods at Montana Junior High School. It appeared that the more hands-on, inquiry approach 
to teaching science positively affected students’ laboratory skills. The effect size chart revealed 
positive educationally notable changes for the hands-on and written full investigations in all 
years but one and for the lab skills performance test in all years except for one. 



Figure 37 

Achievement Scores - Montana 
Student Means 
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Figure 38 




Montana 




Achievement Effect Sizes 
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Summary of Montana Junior High School 

In summary, Montana Junior High School made a very careful and concerted effort to 
implement SS&C. The initial intended curriculum was only slightly modified to fit the context. 
Even though NSTA intended all 9 th grade classes to use the curriculum, at Montana Junior High 
the teachers and administrators thought that students and parents should have a choice between 
the reform curriculum and the traditional courses. As a result, they added SS&C to their list of 
possible courses, rather than using it to replace courses. The curriculum was also slightly 
modified to fit the resources provided by the school. For many activities the science department 
did not have the necessary equipment, so they modified the activities to fit the resources. The 
two 9 th grade teachers and the liaison worked diligently to implement SS&C as close to 
intended. 

Parents, students, teachers and administrators positively viewed the first enacted curriculum. 
Since parents and students had a choice about which class to take, the perceptions of parents and 
students were critical to the continued use of SS&C. As it was, the student outcomes were 
perceived as positive, and interest and enrollments increased during the second year of SS&C. 
External pressures played a very small role in the SS&C curriculum. The State of Montana did 



O 

ERIC 



85 



90 



not institute any mandates in curriculum and assessment, and overall gave schools and districts a 
lot of freedom in this area. This stability allowed the teachers to organize materials for each 
activity, share the same materials, and in the end develop a very complete curriculum that could 
be consistently implemented by different teachers. 



IX. NEW YORK HIGH SCHOOL 



Detailed information about this site is contained in prior reports. What will be presented here is 
a description of the implementation effort at the site in terms of the nature of the community, 
school, and the students, the teachers involved in the imple mentation, the liaison person, the 
emotional and material support provided by the school, external pressures and the progress of 
the implementation effort. The numbers of students participating at this site was small initially 
and decreased as the study continued so the quantitative results should be viewed with caution. 

Nature of the Community, School and Students 

During the study, New York High School was in a lovely setting with extensive grounds 
including a pond and surrounding hills and trees. The site housed both the middle school and 
the high school. The high school building was adequate and generally clean but not classy, new 
or “tip-top”. Stairs had broken edges, doors were worn, and connections between buildings had 
a long term temporary feel, etc. Science classrooms, however, were full of materials, posters, 
equipment, and projects. 

The community was mixed with very high- income homes and very low- income housing 
developments. The low- income segment of the community was growing fast. Most of the low- 
income group is Hispanic/Latino American. There was also a stable, long-term resident 
population of African Americans. Most of the people worked in New York City. The school 
was well supported by both parents and businesses in the community. 

Because of the New York State Regents’ course programs, SS&C had only been available to a 
limited number of students at the high school, approximately 60 students per year. These were 
the students who did not enroll in the New York Regents courses — those who were ESL, those 
with low levels of motivation or those who were academically challenged in other ways. The 
classes over the years were kept small (e.g., 15 students) so that individual attention could be 
provided. All science classes were 50 minutes per day with two extra periods a week for 







laboratory experiences. Freshmen beginning in the 1999-00 school year were required to take at 
least one Regents science course to graduate. 

Nature of the Teachers 

There were approximately ten teachers in the science department over the data collection years. 
The teachers rotated through teaching 9 th grade science so a different set of one to four teachers 
were involved each year. While SS&C was being used, teams of four teachers representing the 
different content areas (earth, physical, chemical and life sciences) taught the course. A total of 
five teachers with a mix of experience teaching attended the training offered during the 
summers. Before becoming involved in the SS&C project, the teachers were content with their 
teaching styles and abilities. Although most were willing to try new things, the experienced 
teachers felt that they were already close to being master teachers. The new teachers involved 
in the reform were interested in learning new techniques. They did not have existing styles to 
change, and their recent pre-service training was related to the standards. This impression was 
supported by the fact that the school had very high standards for hiring and keeping science 
teachers, and the department chair was heavily involved in national science reform. 

Nature of the Liaison 

The liaison for this site was also the science department chairperson. Although he did not teach 
SS&C initially, he did teach it the second year. He had a strong personality and was committed 
to having the school offer SS&C. He interceded with the principal about SS&C and made sure 
the teachers' schedules provided sufficient time to deal with the new curriculum. He also made 
sure there were sufficient supplies. 

Support Provided by the School 

The school was very well equipped in terms of laboratory rooms, science materials and 
computers. Science classes were always held in laboratory- equipped rooms. The science 
department also had a laboratory assistant who gathered or made any necessary supplies and 
helped set up science activities. The teachers all had their own desks in a teacher room and 
often additional office type space in laboratory storage rooms between science classrooms. The 
school was supportive of standards-based instruction and of high achievement for all students. 

To accomplish this the school had strict attendance policies with personalized follow-up. There 





was a lot of teacher- parent interaction about individual students. All teachers were encouraged 
to engage in professional development. New teachers were carefully observed and mentored 
until they received tenure. 

External Pressures 

The Board of Regents controlled curriculum in the State of New York. At the beginning of the 
project the students in the SS&C classes only had to pass a general exam. By the time the 
project ended all students in the State were required to take at least one Regents' science class. 
The changing external requirements meant that the SS&C class as initially configured would not 
meet State graduation requirements. 

Progress of the Implementation 

The project began with the entire 9 th grade, non-Regents students, in a "Topics" science class. 
Topics was mostly an earth science class with general science topics reviewed at the end of the 
course to help the students prepare for the general exam. One teacher taught all of the Topics 
sections. A team of four teachers taught the first year of SS&C. Students spent two weeks with 
one teacher then two weeks with another and so on throughout the year. The second year of 
SS&C the students again had SS&C taught by a team of teachers, but this time they spent one 
month with each teacher. The third year students switched teachers once a quarter. The fourth 
year SS&C was not really taught; students took Active Physics or Chemistry and Community 
for the entire year. The fourth and fifth years of SS&C students took Active Physics. The 
implementation effort appeared to have the most effect on the newer teachers. The 
enhancement training solidified and operationalized the pre- service training the teachers had 
received. Additionally the NRC Standards- based philosophy of SS&C matched with the 
philosophy of the science department so the teachers were mentored into a compatible 
philosophy. 

Based on the teachers' reports of what topics were covered each year, the first year of SS&C 
showed a decrease in the amount of time teaching earth science and an increase in the time spent 
on physics, chemistry, and biology. After the first year the time spent teaching the four aieas 
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was more even for two years. Then the time spent changed to reflect the "pure" chemistry and 
physics courses that were taught instead of SS&C. (See Table 105 in the Appendix.) 

New York High School Learning Environment Measures 

The learning environment was assessed in four different ways. The students and teachers 
completed a questionnaire that asked about what activities occurred in the class. These 
activities were grouped into factors. On the student questionnaire there were four factors: 
inquiry, group work, traditional, and other. On the teacher questionnaire there were two factors: 
inquiry and traditional. The introduction to the individual site sections describes these factors 
and their analyses in detail. The third measure was observations of the classes by external 
observers. These observations allowed the class activities to be categorized into student- 
centered, teacher-centered, group, administrative and other. The fourth measure was a learning 
environment inventory with several scales related to the classroom psychosocial environment, 
which was completed by the students. 

When forced into percent of total class time categories, the student questionnaire data showed a 
significant increase in the percent of time spent in group work and a corresponding significant 
decrease in inquiry the first year of SS&C, whichcontinued through all years of the study. The 
ratings of the scales showed that the students began by rating inquiry at several times per week 
and decreased to between several times and once a week. Group work began at slightly less 
than once a week and moved up to several times a week. 
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Figure 39 

Student Questionnaire - New York 
Percent of Time Spent by Class Activities 
Student Means 
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The teacher questionnaire data were not forced into percentages like the student data. Only the 
frequencies are presented below. There were no significant changes. The scores ranged from a 
low of less than once a week for the traditional activities to a high of more than several times a 
week for the inquiry activities. 



Figure 40 

Teacher Questionnaire - New York 
Frequency of Class Activities 
Teacher Means 

= Inquiry - - - Traditional 
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During the last three years of the project, questions were added to the teacher questionnaire. 
Teachers were also asked to rate the percent of time they spent in whole class activities, group 
activities, and individual activities. After doing these ratings, the teachers then described what 
happened within each category. These data showed that whole class activity increased and that 
it was mostly due to increases in demonstrations. The teachers also reported that time in 
individual activities decreased, and the mix of individual activities changed to include more 
journaling, computer work, and worksheets. (See Tables 1 12-1 14 in the Appendix.) 
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The classroom observation data showed a significant change toward a more student-centered 
environment at the beginning of SS&C and then a decline and then a resurgence this past year. 
Student- centered behaviors changed from 10% of the time to about 50% of the time and then up 
to 75% percent of the time after a brief decline to 20% in 1999. These changes were mirrored 
by opposite changes in student- and teacher-centered behavior. 



Figure 41 

Classroom Observations - New York 
Percent Time Spent by Teaching Style 
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As a second look at the classrooms, the observation forms asked the observer to rate the level of 
student involvement and type of instruction. These ratings showed no significant changes 
except for student involvement in 2000 and type of instruction in 1999, but showed a trend 
toward less student involvement and more inquiry- oriented instruction. 



The scales on the learning environment inventory were mostly stable except for a significant 
increase in sequencing the first two years of SS&C and then a slight decrease back to original 
levels until the last year when sequencing increased significantly again and difficulty decreased 
significantly. 



Figure 42 

Learning Environment Inventory - New York 
Student Means 
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The changes seemed to be directly related to the SS&C enhancement effort in the first two 
years, and then environments became highly dependent on the individual teachers involved in 
the Topics classes. At least part of the increase in student- centered activities the last years was 
due to the nature of the students in the classes. Over the years the proportion of special 
education students in the classes increased, and along with this came the need for individualized 
instruction. Additionally the curriculum used in the last years was designed to be student- 
centered. 
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New York High School Student Outcome Measures 

The student outcome information is grouped in two ways. The inclination of students to 
continue studying science and student motivation and attitude toward science is one group, and 
student achievement on the science literacy multiple choice items, science literacy open-ended 
items, lab skills performance test, and written full investigations is the second. There were only 
a small number of students involved in the classes at New York High School. Therefore 
differences must be quite large to be statistically significant and averages are much more subject 
to influence by individual scores. 



Attitude & Motivation Measures 

There were few changes in student inclination to study science, no doubt because of the strict 
and continuous State requirements. There was a significant increase and then decrease in the 
students wanting to pursue a career in science in 1 999 and significant decreases in 1 998 and 
2000 in students believing science activities made them want to take more science. This may be 
due to the increasing proportion of special education students in the course. In 1998, there was 
a significant drop in “finding science class motivating” and "science class is interesting," but 
these returned to comparison year levels. 



Figure 43 

Student Questionnaire - New York 
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Figure 44 

Student Questionnaire - New York 
Science Motivation 

Percentage of Students who Responded "YES" 

- - - Career in science? — * “ Is your science class motivating? 

•••■■ Activities lead to more science? Had a "totally awesome" experience in class? 
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Achievement Measures 

There were some changes in student achievement in keeping with the changes in classroom 
environment. The lab skills performance test, the written full investigation and the science 
literacy open-ended items showed significant increases the first two years of SS&C. These 
differences continued into 1998 for the science literacy open-ended items and the lab skills 
performance test. The lab skills performance test, the science literacy open-ended items, and the 
hands-on full investigation showed significant differences from the comparison year in 2000. 
The differences for the first three years on the lab skills performance test were due mostly to 
differences on the rock identification and instruments tasks. These differences seemed most 
related to the increased and continuous use of student laboratory activities in the classes in 
contrast to the comparison year. Even though SS&C was not used, the Active Physics that 
replaced it is also a very hands-on curriculum. The effect size chart revealed positive 
educationally significant changes for the labs skills performance test and science literacy open- 
ended questions in all but one year. There were also positive educationally significant changes 
for the hands-on full investigation (1997, 1999 and 2000) and the written full investigation 
(1996-1998). Negative educationally notable changes occurred for the written full investigation 



Figure 46 

Achievement Scores - New York 
Student Means 
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Summary of New York High School 

In summary the New York High School implementation was incomplete. Considering their 
experience in light of the proposed model for teacher enhancement highlights several crucial 
components. Initially the school was not wholly ready to participate in the implementation. 

The school was well equipped to implement the curriculum. The principal and the department 
head were very supportive; the teachers were well qualified and believed in inquiry-based 
science instruction; the school had all of the physical resources necessary for the curriculum, 
and in general the community and students were interested in trying something new. On the 
other hand, although some teachers were interested in the effort, others had mixed feelings or 
were actively opposed to the curriculum. Many believed what they were doing already was 
better than anything the new curriculum would be able to provide. 

The first enacted curriculum reflected the skill and perceptions of the teachers at the school. 
They rearranged the curriculum to better take ad\antage of their instructional skills and content 
expertise and implemented it in that format using teams of teachers. The classes were very 
student-centered and hands-on, and student attitude and achievement went up slightly. Not all 
teachers' perceptions about the curriculum, however, were positive. At times attitudes toward 
the curriculum were based more on attitudes toward the liaison teacher who was supportive of 
the program than the curriculum itself. Because of these mixed impressions and the new sets of 
teachers involved each year, the curriculum was continuously modified. Classrooms stayed 
more student-centered, and lab skills stayed higher for a few years. No teachers had the course 
as a continuous responsibility, and it was difficult to coordinate the efforts of the team of 
teachers. Eventually the course devolved into being taught by one teacher. In addition to the 
continuous changes due to the different teachers, the State and the school changed their support 
of the course. At first the course was an acceptable one for graduation; however, during the 
course of the study the rules changed and the course was no longer acceptable. Therefore the 
value of the course was significantly decreased. The prestige was further diminished through 
the administration's rescheduling of the students who could take the course so that the mix of 
students became proportionately more special education students. By the fourth year of the 
study nothing of the original project was left except in the attitudes of some of the newer 
teachers who had participated in the training. Completely different curriculums, although also 
Standards-based, were being used in the classes. 
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X. TEXAS HIGH SCHOOL 



Detailed information about this site is contained in prior reports. What will be presented here is 
a description of the implementation effort at the site in terms of the nature of the community, 
school, and the students, the teachers involved in the imple mentation, the liaison person, the 
emotional and material support provided by the school, external pressures and the progress of 
the implementation effort. 

Nature of the Community, School and Students 

During the study, Texas High School was near the downtown of a large city and only blocks 
away from two Universities. The surrounding urban area was small tract housing or larger three 
story walk-ups. The residential area was somewhat mixed with larger single family homes 
further away from the school. The community ranged economically from very low to middle 
class. The student population was almost entirely African American with a few Asian American 
and Hispanic/Latino American students. The community and the population of the school were 
stable over the course of the study. 

The school was housed in an older traditional style three-story brick/cement block building in a 
“U” shape. The building was generally clean but minimalist and stark in appearance. The 
parking lot was fenced in, and the classrooms were kept locked. Staff were assigned to watch 
the halls, but the spirit was more one of concerned parents keeping track of their children than 
policing. The science classrooms were grouped together in one hall. Most science teachers had 
two classrooms in the hall, a traditional classroom with moveable desks and a science laboratory 
room. Ninth grade science class sizes over the years ranged from 15-34. Classes were offered 
on a 90- minute block schedule with different classes on alternating “A” and “B” days. 

Nature of the Teachers 

There were approximately 1 1 science teachers in the school each of the data collection years. 
The teachers usually taught in their areas of specialty, so of the 1 1 science teachers, three to 
seven taught from two to six sections of 9 th grade science each. There were only two teachers 
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who taught 9 th grade science every year. Other slots saw a lot of turnover. There were several 
teachers involved in the different training opportunities. Each year, however, several new 
teachers were responsible for teaching SS&C. There were two teachers who attended the 
training who were involved in teaching SS&C for the entire period of the study. The two long- 
term teachers had been at the school for many years before SS&C began. They were both 
mainly physical science teachers and personally interested in their students' success. One was 
also particularly interested in hands-on activities. The new teachers were necessary because of 
changing external requirements and because of teacher turnover. Hope for, and belief in, the 
students was high among the teachers. Some of the science teachers were former Texas High 
School graduates who came back to share their success and understanding with the students. 

The students were generally polite and were well behaved when kept occupied. There was an 
acceptance of school, and the attendance rate was 92%. 

Nature of the Liaison 

The liaison for this site was the science department chairperson for the first two years of the 
project. The third year she resigned from her position as chairperson, and then she retired. One 
of the long-term teachers then became liaison teacher. The first liaison teacher was very 
supportive of SS&C and made sure the teachers' schedules provided sufficient time to deal with 
the new curriculum and that the district provided the necessary supplies. She was available for 
questions and actively lobbied the principal to support the SS&C implementation. The third 
year she still was supportive of SS&C and tried to help the newer teachers in their 
implementation, but she was no longer in a position of direct power. The fourth year the new 
liaison teacher was also the 9 th grade curriculum coordinator. 

Support Provided by the School 

Based on interviews, the principal appeared to be astute politically and concerned about 
enhancing the reputation of the school. The district was quite large and the principal felt that he 
was responsible for implementing many of the District’s and the State’s constraints on 
instruction and curriculum. Over the years the principal’s interest in SS&C changed. He was 
quite supportive in the first years when the project was funded and had a national focus, but that 
dwindled to what appeared to be a neutral, apathetic attitude when the funding was cut. There 



were few supplies at the school, but what could be provided was available to the teachers. The 
district was neutral in terms of the SS&C curriculum. Although the 9 th grade teachers were part 
of the entire science department, they were a fairly distinct and separate group. 

External Pressures 

Although individual school districts in Texas had some choice in the curricula they used, the 
State provided an approved list. At the beginning of the project there was no required 9 th grade 
curriculum. By the time the project ended all students in the State were required to take 
physical science in 9 th grade. The changing external requirements meant that the SS&C class as 
initially configured would not meet State graduation requirements. It also meant that teachers 
with a biology certification could not teach the class even if it retained some biological 
components of SS&C. 

Progress of the Implementation 

The project began with the 9 th grade students taking a variety of science classes although most 
were in some sort of physical science. When SS&C began all 9 th grade students were enrolled. 

* t h 

This continued for three years. After the third year of the implementation the State required 9 

t h 

grade to be physical science. The district took the State guidelines and developed its own 9 
grade science course that all students were then required to take. The second 9 th grade liaison 
teacher was part of the district team developing the new 9 th grade physical science course. 

Based on the teacher's reports of what topics were covered each year, the first year of SS&C 
showed an increase in the time spent teaching biology and earth science and a decrease in the 
time spent teaching chemistry and physics. After that first year, the amount of time spent 
teaching the four science areas stayed fairly even until 1999 when the classes became only 
physical science (physics and chemistry) once again. (See Table 131 in the Appendix.) 

Texas High School Learning Environment Measures 

The learning environment was assessed in four different way s. The students and teachers 
completed a questionnaire that asked about what activities occurred in the class. These 
activities were grouped into factors. On the student questionnaire there were four factors: 
inquiry, group work, traditional, and other. On the teacher questionnaire there were two factors: 
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inquiry and traditional. The introduction to the individual site sections describes these factors 
and their analyses in detail. The third measure was observations of the classes by external 
observers. These observations allowed the class activities to be categorized into student- 
centered, teacher- centered, group, administrative and other. The fourth measure was a learning 
environment inventory with several scales related to the classroom psychosocial environment, 
which was completed by the students. 

When forced into percent of total class time categories, the student scales showed that there 
were significant increases in the percent of class time spent on group work and decreases in the 
percent of time spent on other and inquiry activities, which continued for the first three years of 
the enhancement effort. The group work began at slightly less than once a week and moved up 
to between once a week and several times a week. Despite the decrease in the percent of class 
time spent on inquiry, inquiry activity increased from about once a week to slightly higher than 
that. 



Figure 48 

Student Questionnaire - Texas 
Percent of Time Spent by Class Activities 
Student Means 

Inquiry - - - Group Traditional Other 




Comparison 1995 SS&C 1 1996 SS&C21997 SS&C3 1998 SS&C41999 SS&C 5 2000 
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The teacher questionnaire data were not forced into percentages like the student data. Only the 
frequencies are presented below. There were no significant changes. The scores ranged from a 
low of slightly less than once a week for the traditional activities, to a high of more than several 
times a week for the inquiry activities. 



Figure 49 

Teacher Questionnaire - Texas 
Frequency of Class Activities 
Teacher Means 

= Inquiry - - - Traditional 




Comparison 1995 SS&C 1 1996 SS&C 21997 SS&C31998 SS&C41999 SS&C 5 2000 

During the last three years of the project, new questions were added to the teacher questionnaire. 
Teachers were also asked to rate the percent of time they spent in whole class activities, group 
activities, and individual activities. After doing these ratings, the teachers then described what 
happened within each category. Based on these ratings, the amount of whole class activities 
decreased, and individual activities increased. The mix of individual activities reported showed 
a decrease in lab write-ups and an increase in text reading and other activities. (See Tables 138- 
MO in the Appendix.) 



The classroom observations showed a significant decrease in teacher- centered behavior with 
significant increases in student-centered behavior for the first two years of SS&C and then 
significant increases in teacher-centered behavior again. Student-centered behavior ranged from 
a low of about 10% to a high of 40% and back down to about 10%. Student- and teacher- 
centered behavior increased from 10% to 40% in 1997 and remained there. 



Figure 50 

Classroom Observations - Texas 
Percent Time Spent by Teaching Style 

- - - Teacher-Centered Administrative 

Student- & Teacher-Centered Other activities 

— ■ - Student-Centered 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 

As a second look at the classrooms, the observation forms asked the observer to rate the level of 
student involvement and the type of instruction. These ratings showed no significant changes 
but showed a trend toward less student involvement in later years. 



The learning environment scales showed continuous significant increases in relevance and 
involvement and significant increases in experimenting for the first three years. The other 
scales showed mixed results or stability. 



Figure 51 

Learning Environment Inventory - Texas 
Student Means 

— - Difficulty Involvement 

Relevance Sequencing 

™ ■ ■ Critical Voice ■■■■ Experimenting 




Comparison 1995 SS&C 1 1996 SS&C21997 SS&C31998 SS&C41999 SS&C 5 2000 

The changes in the learning environment seemed directly related to the SS&C enhancement. 
Despite new teachers, they tried to consistently use the curriculum in the intended ways for the 
first few years. After those years the implementation was very teacher specific with most 
teachers doing what they felt was best. With the implementation of the new physical science 
class the individualization increased even more. Within the individual styles fewer teachers 
chose to use hands-on activities. 
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Texas High School Student Outcomes Measures 

The student outcome information is grouped in two ways. The inclination of students to 
continue studying science and student motivation and attitude toward science is one group and 
student achievement on the science literacy multiple choice items, science literacy open-ended 
items, lab skills performance test, and written full investigations is the second. 



Attitude & Motivation Measures 

The initial increases in student-centeredness and the use of more hands-on activities in the 
classes appeared to contribute to some changes in student attitudes. There were significant and 
sustained differences in students finding their science class motivating, and students reported 
significantly more "totally awesome" experiences for the first three years. There were also 
significant increases the first year in students wanting to pursue a career in science and in the 
first two years in wanting to take more science. Students found their science classes 
significantly more interesting than the comparison year for the first three years and significantly 
more fun for the first four years. 



Figure 52 

Student Questionnaire - Texas 
Inclination to Continue Studying Science 
Percentage of Students who Responded "YES" 

“ “ “ Take science in 10th grade “ ■ " Take science in 12th grade 

Take science in 11th grade Take science in college 




Comparison 1995 SS&C 1 1996 SS&C2 1997 SS&C31998 SS&C4 1999 SS&C 5 2000 



Figure 53 

Student Questionnaire - Texas 
Science Motivation 

Percentage of Students who Responded "YES" 

■ ■ ■ Career in science? “ ■ ■ Is your science class motivating? 

Activities lead to more science? Had a "totally awesome" experience in class? 




Comparison 1995 SS&C 1 1996 SS&C21997 SS&C31998 SS&C 4 1999 SS&C 5 2000 





Figure 54 




Student Questionnaire - Texas 




Attitudes Toward Science 


■ ■ ■ Science is useful 


Science in necessary to get a good job 




■ Science class is interesting 


“ • “ Science will be useful in the future — Science class is fun 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 
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Achievement Measures 

Despite the changes in student attitude, there were few effects on student achievement. The 
SS&C students had significantly higher scores than the comparison year students on the lab 
skills performance test the first and third years of the implementation. These effects were 
mostly due to scores on the rock identification station. The effects in the second, fourth and 
fifth years on the literacy test and the fourth and fifth years on the science literacy open-ended 
items showed significant decreases in contrast to the comparison year levels. The decreases on 
the literacy test showed up for almost all of the scales. The effect size chart revealed positive 
educationally significant changes for the lab skills performance test (1996, 1998 and 2000), the 
hands-on full investigation (1999, 2000), and the written full investigation (1997). Negative 
educationally significant changes occurred for the written full investigation (1996, 1999 and 
2000), the science literacy open-ended questions (1999 and 2000), and the science literacy 
multiple choice items (2000). 



Figure 55 

Achievement Scores - Texas 
Student Means 

Percent Correct on Achievement Tests 

- - - Lab Skills Performance Test Written Full Investigation 

Science Literacy Multiple Choice Items Science Literacy Open-Ended Questions 

“ ■ ■ Hands-On Full Investigation 




Comparison 1995 SS&C 1 1996 SS&C 2 1997 SS&C 3 1998 SS&C 4 1999 SS&C 5 2000 
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Figure 56 
Texas 

Achievement Effect Sizes 

• • • Lab Skills Performance Test Written Full Investigation 

■■■■■ Science Literacy Multiple Choice Items Science Literacy Open-Ended Questions 

™ ■ * Hands-On Full Investigation 




1995-1995 1995-1996 1995-1997 1995-1998 1995-1999 1995-2000 



Summary of Texas High School 

In summary the Texas High School implementation was incomplete. Considering their 
experience in light of the proposed model for teacher enhancement highlights several crucial 
components. The school was not wholly prepared to implement the curriculum. The faculty, 
and especially the liaison teacher, were committed to it and felt it would really help their 
students. The principal and the school district were supportive. There was not appropriate 
equipment to support the implementation, and not all the teachers were qualified to teach it. 

The first enacted curriculum directly matched the intended curriculum in terms of activities to 
be covered. The teachers were hampered, however, by the lack of equipment and materials. 
There was additional teacher training provided by a nearby University. The classes were more 
student-centered and hands-on. The second year of the implementation new teachers were hired 
to teach the curriculum. They had some training but not as much as the prior group. Only two 
teachers from the prior group remained. The curriculum was still implemented as planned. The 
liaison teacher was quite active in supporting the new teachers, and the administration was very 
supportive. The third year there were once again new teachers teaching the curriculum and even 
fewer resources since the funding for the implementation project had ceased. The liaison 
teacher was no longer department chair and was less influential in supporting the curriculum. 



This year the curriculum was implemented in a more individualized fashion with each teacher 
doing what he or she felt best and with overall increases in traditional instructional styles. This 
trend continued until the end of the evaluation project. In the last years the State required that 
the 9 th grade class be physical science rather than the coordinated science of the implementation 
project. In addition the State instituted a high stakes test, and much of the science class time 
was spent in directly preparing students for the material on that exam. These requirements 
meant that most of the curriculum was abandoned except for some physical science activities 
and some changes in the minds of the two remaining instructors. 



0 

ERIC 



118 



Evaluating the 
Long Term 
Effects of 
Teacher 
Enhancement 



o 

o 


s 

<D 

£ 


CN 










GO 


cd 


<D 


g 


O 




S 


<d 




[Jh 





§ 

S 



4h 


V 


K 


'3 


00 


% 

hJ 


’oh 


*£ 


O 


<D 


Q 


PQ 



<4-H 

o 

<5 S> 

> s 

’£ a 
D 2 



1^. 

V 



O 

ERIC 



M m ^ h oo 



on o 



c /5 

C 

a/ 

c 

o 

U 

O 

3 

C 3 

H 




<N fO 



^f »/N 



xA 

$> 
TJ 

•S -5 -5 Jz 
s s e 5 

C§*C§*C§* <$ 
V V V B 

.§ .5 .5 O 

g 

-*■*»•*■**•*•*»/•, 
K K K 3 

Qj ^ *> > 

<o <o <o ^ 

X. X. 5 < 

cu cu ea 

^O 

CG 

.S 

*3 

s 



</3 

$> 

H 

$> 

u 

c 

Cl 

s 



*9 

^ e 
S -2 
£ t 8 
^ a 
-M .« O) 
8 2 ■* 
H§-§ 

cg 

u ^ K 
^ ***** 

.■2 5 6 , 

3 ^ O' 

o> 

o 

s 

.2J 
*3 
c n 




GO 

v*H 



O 

ERIC 



r- oo 



5/5 

« 

'EL 

o 

H 

© 

© 

c 

. 2 i 

’3 

03 

c 

o 

c 

© 

Q. 

03 



© 

© 



5 £ 

E 2 
s s 
Z 



© 

1 fl 
s ^ 
.2 ^ 
* s 

,2 B 

s 

’3 o 
© ,*- 
> > 
- H 
E y 



C\ o — m 

<N m m m 



<N 

m 



o 

3 

co 

IP 

4 

3 

Co 

3 

3 



^ ^ h 00 ^ O H 

<T) cn <T) r) r) n Tf Tt 



§ .2 
O a 
© o •£ 

L« 3 ^ 

'111 

11 1 

5/5 ^ 

© O •** 

a g* 

a ■ ' <=y^ ^ 
J 2 euo^ 
a fl +- 
c 
© 

m T 3 
U h-3 03 



Co 

J3 

u 

*3 



3 

o 

co 

So 



<o 

o 

3 

•Si 

co 



^3 

. fij 

s s 

3 -Sh 

^ SJ 

c§ ’£ 

7? 3 

3 O 

fl 

CO f? 
^3 



3 

— ^ 
>; 53 

£ ^ 
R 3 

^3 

3 ^ 
r 3 i ^ 

Co 



>5 

•Si 

•5 03 

> -Si 
-3 

^ *3 

Co 

§ ^ 

?-* >J 

§ U 

^ *3, 

> ST 

^ 2 

►3 e> 

e c 



> CA 
•~ U 

o H 

^ = 

n < 

•S a 

O o 

-2 | 

g? fc 

S: O 

e W 



C K C 

Co 1 ^ Co 1 

QJ * 4 J <^> 



■r h 

*- u 

a 
© 



© 

u - - 

!S .§ .§ 
a ^ ^ ^ 
E ^ ^ -*«* 
0 3 3 3 

*C Qi Qi Qi 

<zj 

© ^ ^ 

3 <4) ^ 

^ ^( 

Im 

o 

J= 

w 

a 

o 

H 



g 8 
g H 

£ 8 

C 

5 ^T a 
o> s 
>- g 

© £ 
03 u 
^ © 
C &H 
© 

E 

Z 03 

IE xt 
© a 
◄ J 



5/5 

o 

H 

© 

© 



W o 
o ’3 s 
-r a o 
3 m *C 
•** a 

t/5 CJD 
3 rti •« 

si* 

^ a © 
>>^h > 

2 s « 

a - nn 



= J 



© 

w 

s 

© 



fe 

s 

O 



5/5 

-o 

c 

03 n 



3 

fe 

C 

<U 



r- 

V© 



<N 

Tf 



<n co in vo r^ 

Tf Tt Tf Tt -Of Tf 



>o vo r^ on 

^ (O irnn 




+* 


u 


c 


O 


© 


X 


-o 

3 

03 


© 

a 

o 

H 




o 

ERIC 



'O 'O 'O 



^ (N ^ 
OO OO 00 oo 



50 

.<x> 



•R >} 
^ .*> 



'O r^- oo ^ o h 

OO OO OO oo ON 



J3 

<j 

Q 

:f 

"3 

JC 



50 

§ 

****** 

<j 

§■ 

2 

e> 



R R r 



r 

Cl 

•- I 

cs .5 

C ^*H 

s - 

o ss 
!£ ^ 
CO O 

g 

O' 0 " 

u 

4> 

J3 

CJ 

« 

<U 

H 



•s s 

X5 H 

^ = 

«o ^ 
52 ^ 

■S ss 

<J © 



u 

a> 

u 

u 

© 

U 



CO 

© 

H 

© 

CJ 



R R 3 

Qi Qi o 

A A k- 

Co"Co" <U 

^ ^ Pm 

,| S 

Sfi g 

^ ^ O 

R R 3 

<L r/} 
U U 

^ ^ t: 

<4) <4J C 

^ | 
© 
> 
4> 



4> w 
H « 

<y Cm 
£2 
* 5 

I ^ , 

o 

C Ci 
Cl 



c 
o 
•& c 
o 

.2f m & 

03 

2 -2f 

'a? 

C £J 



CJ 

< 



O 

Pk _ 

£ 3 

<U 
U 

s 
<y 



0> Mh — 



C/3 

-O _ 
C$ o 

ij 1/3 



fe 

s 

9 

CO 

*© 



s 

© 



C3 > 

a £ 




o 

ERIC 



ON ON 



S3 

© 



08 © 
WT-C 
tc C8 

g .Sf 

> W5 

C 4> 

HH > 

— c 
a ~ 

= = 
O c 

l S3 
M © 







M Z. 


u 


© 


05 5 
o £ 


© 

pis 

U 


© 

u 

C/i 

V3 


> 1 


C8 

© 


cs 


> 55 


H 


□ 


w 








IT) 

Tf 



o 

<N 



O 

CN 



hJ 

o 

o 

a 

u 



>5 



o 

oa 

§ 

X 
£ 
© S 

i£ 



g 2 

.2 R 
c ^ 

w O 

« § £ 

a o' ^ 

a z; 

a s 
Si 

x 5 
w 

H 



CM 

H 



ERIC 



^ (N m Tt in 
CM CM CM CM CM 



m ^ m 
m m m m 




OO Os O 
m m 



H (S ^ ^ 

Tt Tt Tt Tt Tt Tt 



&a 



«o 

<4> 



-5 

<J 

I 

"3 

K 






s: 



>> 

J3 

O 

§• 
s 
o 

si Si 



Q fl 

_ 4> 

3 H 

^ = 

Co ^ 

S ^ 

J3 c 



s: s: 
„ ^ 



u 4) <4> 4) 

« .§ .§ .§ 

S ^ ^ f'i 



co 

CJ 

3 

O^ 

i- 

4> 

JS 

w 

fl 

4) 

H 



s: 

S 



c c 

4> 4) 



g *. 

4) 4> 4) 

^ ^ a* 



4> 

0> 

L. 

J- 

o 
U 
+■» 

g 8 

4) 4> 

* « 



CO 

4> 

H 

4) 

4> 



u § 

4) VS 
7S fl 
.£- WD 
M 



§ ► ts 



>» « 

4> — 

fl "fl 

J- r s 

4> ^ 



S = nj 



50 

4> 

.y 

a £ 

.2 ^ 

+* 4> 

fl .4) 

m sc 

CO ” 
4) 4) 

s 

o 

4> 

_ fl 

fe o 



5 c» 

Ifl X 

4> fl 

◄ iJ 



9 «: 



CO 

-a 

a 

ffl 



c 

4> 

-a 



£5 



ERjt 



APPENDIX B 147-162 

REFERENCES 163-164 



Appendix A 
Tables of Results 
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Table 1 

Student Questionnaire 
Student Means 

Percent Time Spent on Inquiry Class Activities 
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Table 2 

Student Questionnaire 
Student Means 

Percent Time Spent on Group Class Activities 
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Table 3 

Student Questionnaire 
Student Means 

Percent Time Spent on Traditional Class Activities 
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Table 4 

Student Questionnaire 
Student Means 

Percent Time Spent on Other Class Activities 
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Table 5 

Student Questionnaire 
Student Means 

Frequency of Inquiry Class Activities 
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Table 6 

Student Questionnaire 
Student Means 

Frequency of Group Class Activities 
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Table 7 

Student Questionnaire 
Student Means 

Frequency of Traditional Class Activities 
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Table 8 

Student Questionnaire 
Student Means 

Frequency of Other Class Activities 
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Table 9 

Classroom Observations 

Percent of Time Spent in Teacher-Centered Activities 
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Table 10 

Classroom Observations 

Percent of Time Spent in Teacher- and Student -Centered Activities 








O 




© 








a 


in 


© 


ON 


© 


© 


© 

© 






V 




V 


V 


© 














CN 














m 




r- 


© 


CN 


ON 


1— < 


t n 


N 


NO 




i— | 


ON 


© 


m 

on 




© 


1 


© 


Os 

1 


cn 


as 


< 


T d" 

r- 




^d - 

cn 


ON 

cn 


^d - 

© 






in 


m 

i 




NO 

n 

i 


NO 

cn 






oo 


ON 


© 




r- 




a, 


CN 


CN 


NO 


© 


© 


Os 








' 






as 














as 
































r- 


NO 


cn 


^d - 


i— i 


t/5 


N 


© 


© 


in 


NO 


oo 










© 


cn 


1—4* 


in 




1 


1 




i 




on 














on 




cn 


r- 


oo 


oo 


r- 




< 


CN 


r^ 


so 


CN 


CN 






i— ' 


ON 


NO 


in 


NO 






1 


i 


cn 


CN 






ON 




© 




SO 




a 


CN 


© 


IN 


© 


© 


00 

Os 






V 




V 




as 














r-H 


















NO 


i—i 


ON 


ON 


ON 


on 


N 


© 


NO 


cn 


1-H 


OO 


> 




_ 4 * 


cn 


© 




1—4* 


m 




i 


i 


i 


1 




ON 














ON 




in 


oo 




Os 


OO 


rH 


< 


cn 


00 


CN 


00 


NO 






i—i 


© 


cn 

i 


^d 1 


OO 






i 


in 

i 


m 

i 


CN 






o 


© 


© 




cn 




tx 




© 


r— 1 


© 


© 


r- 

ON 










V 




ON 














rH 


















Os 


i— t 


© 


rt 


cn 


V!) 


N 


cn 




^d; 






<n 

ON 




© 


^7 


» 


cn 


CN 


ON 




t 


so 


CN 


oo 


cn 


fH 


< 


NO 


NO 


NO 


n 


CN 






cn 


OO 


i— i 


CN 


oo 






CN 

1 


i 


in 

i 


cn 










OO 


r- 


r^ 




Cl, 


r- 


© 


oo 


© 




NO 

ON 






V 








ON 














1-H 


















ON 


CN 


m 


© 


cn 




N 


CN 


CN 


i— 1 


oo 


r- 


> 




© 


cn 


© 


i 


© 


m 






i 




1 




ON 














ON 


< 


OO 

r- 


oo 

ON 


CN 

NO 


cn 


oo 






cn 


CN 

^d - 

i 




CN 

cn 

i 


in 
















O 














o 




























CJ 

vi 




.2 

•g 




cd 


•a 

o 








c 

o 




s 


>H 


c/5 






Jg 

Id 

o 


Iowa 


c 

o 

s 


£ 

<D 

% 


1 

H 







c 


cn 


(N 


cn 


- 


cn 


m 




ai 

00 


NO 


NO 


CN 


© 




u 

«8 


w? 

© 

© 


oo 


m 

© 


OO 

oo 


© 

© 


00 

© 


cn 

cn 


CN 


C 

cd 


(N 

cn 


NO 


n 


© 


(N 






0) 


cn 


in 


NO 


© 


in 






£ 


cn 


© 


NO 


© 


f — 4 








N- 








^d* 






c 


NO 


in 


in 


in 


NO 




ON 

ON 




in 


© 


© 


in 


m 


u 

«8 


ai 

00 


in 

in 


in 

cn 


CN 

© 


ON 

© 


oo 

cn 


cn 

cn 




C 

cd 


NO 


CN 


© 




in 






0) 


cn 


CN 


© 




IN 






£ 


NO 


CN 


CN 


r-i 


1-H 












CN 


N* 


cn 






c 


oo 


cn 


NO 


r- 


oo 


<n 


00 s 

ON 

ON 




© 


CN 


© 


m 


© 


u 

«8 


ai 

00 


On 

in 


NO 


© 


NO 

© 


in 


cn 

cn 


1-H 


Ch 
















cd 

a) 


in 


1-H 


OO 


© 


in 






<N 


1— J 


© 


in 








£ 


NO 


1—4 


CN 


i—4 
















CN 


cn 






£ 


cn 


CN 


cn 


cn 


cn 


CN 






On 




o 


r- 


N- 


u 


K 

ON 

ON 


ai 

GO 


NO 

CN 




r- 

cn 


00 


N* 

N 


cn 

cn 




Ch 
















cd 


i—i 


cn 


© 


i—i 


© 






a> 


CN 


cn 


r- 


NO 


N 






£ 


i—i 


cn 


cn 


cn 


cn 








cn 


cn 


CN 


N" 






cl 


oo 


n 


NO 


NO 


r- 


hH 






N" 


CN 


r- 






u 

*3 


NO 

ON 

ON 


s.e 


ON 


in 

Tt- 


r-; 


CN 

NO 


CN 

NO 


cn 

cn 


fH 


G 

cd 

0) 


OO 




^d- 


r- 


in 






cn 


© 


ON 


© 


© 






£ 


1—4 


ON 


NO 


N* 


© 








cn 






N" 








a 


Os 


NO 


r- 


cn 


r- 


a 
















o 
















Xfl 
















\ 


in" 

ON 








© 


in 


in 


N 

cd 


ai 


Os 


in 


'd- 


NO 


CN 


a on 


oO 


oo 


CN 








a 
















c 

o 

u 


>w ' 


C 

cd 


© 


Os 


(N 


Os 


oo 




a> 


NO 


ON 


cn 


cn 








£ 




1— H 


in 


NO 


in 








CN 


NO 




r- 


















O 
















O 

w 






.2 




cd 


•a 

o 




cn 






E 

o 




s 












jg 

Id 

a 


Iowa 


c 

o 

s 




Texa; 






CO 

tH 



Table 11 

Classroom Observations 

Percent of Time Spent in Student -Centered Activities 
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Table 12 

Inclination, Motivation & Attitudes* 
Percentage of students who replied "YES 
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Table 13 

Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score and Station 



O 

ERIC 







NO 




CO 




NO 




Oh 




o 


p 


o 


O 


o 

o 






V 




V 




o 














N 




ON 


_ 


CM 


ON 






N 


co 


CM 


CM 


OO 


ON 


in 

Os 




i 


of 


CM 


CM 


1 ““ l 


On 




oc 


NO 


r- 


o 


C- 




< 


oo 


O 


of 


p 


NO 






in 


r-H 


ON 




ON 






1 


’ ' 












in 


OO 


Of 


of 


CO 




Oh 


y—i 


o 


O 


NO 


i—i 


ON 














ON 














ON 


















in 


r- 


CM 


c- 




c/5 


N 


of 


t> 


© 


of 


in 


m 

ON 




’T 




CM 


o 




ON 




in 


NO 


Of 


i— H 


i— H 




< 


CM 


m 


Os 


ON 


Of 






NO 


NO 


oo 


CM 


NO 






Of 


00 


oo 




CO 




Oh 


in 


O 


y— H 


© 


o 


00 














ON 














ON 














t-H 






00 


CO 


NO 


CO 




N 


NO 




CO 


t> 


p 


in 

ON 




o 




1 1 


CM 


CM 


ON 






NO 


m 


y—l 


ON 




< 


in 


OO 


o 


O N 


in 






CM 


in 


t> 


OO 


ON 






1 














NO 








O 




Oh 


O 


in 


o 


o 


tp 


On 








V 






ON 














t-H 




in 


NO 




_ 


00 


C/5 


N 


oo 


NO 


t—H 


tp 


co 


>• 

m 

ON 




p 


© 


CO 


CM 


o 


ON 




CM 


of 




o 


CM 




<1 


O 


of 


p 


co 


co 








CM 


CO 


CM 


CM 










*" H 


CM 








CM 


r- 


Of 


CM 


CO 




Oh 


CM 


*— 1 


O 


O 


o 


NO 














ON 














ON 














i—i 




of 


OO 


r- 




in 


Si 


N 


CM 


co 


p 


CM 


»— • 


> 

in 

ON 




p 


, " H 


CM 


CM 


CM 


ON 




o 


CM 


NO 


C- 


C- 




< 


«— 1 


CO 


of 


p 


OC 






in 


of 


00 


00 


ON 






i 






r-H 


















© 














o 














J5 














© 

CZ) 




.2 

■a 






% 

o 








c 

o 




s 












a 


c 


£ 


s 

X 

<D 

H 






"3 

O 


£ 

o 

M 


1 


iu 

2 









On 


o 


in 


r- 


ON 


in 




Ch 


CM 


CO 










qJ 

oo 


i— < 


CO 


^f 


in 


NO 


u 

«8 


o 

o 


CO 


^f 


p 

CM 


CM 


ON 

CO 


cz> 

cz) 


w 


a 


CO 






r- 


00 






cd 


in 


l— 1 


00 


ON 


CM 






<d 

rt 


o 


r-H 


K 


l> 


o 






£ 


NO 


00 


r- 


in 


in 








NO 


o 


cf 


f , 


NO 






Ch 


CM 


CO 


CM 


CM 


CM 


or 


/"■•v 




CO 


o 


00 


O 


CO 


u 

«8 


ON 

ON 

ON 


s.e 


CM 

CO 


p 

CO 


00 

CM 


00 

CM 


00 

CM 


cz) 

cz) 






NO 


CM 


00 


00 


CM 






ctf 


i— i 


NO 


CM 


00 


O 






<D 


O 


ib 


K 


CO 








£ 


NO 


t> 


r- 


^f 


of 








C- 


o 


in 


r- 


00 








CM 


CO 




i i 


CM 


CO 


/■— \ 

00 

On 




ON 


in 


o 


CM 


ON 


U 


<D 


O) 


^f 










ON 


00 


CM 


CM 


^f 


^f 


CO 


cz) 

(Z) 


i-H 


a 


o 


CM 


ON 


00 


O 






cd 


O) 


ON 


CO 


00 


CM 






<D 


CO 


in 


in 


ON 


O 






£ 


NO 


c- 




m 


in 








NO 


NO 


^f 


ON 


CM 










CM 








CM 


fP 


oJ 

00 


^f 


00 


CO 


Of 


, 


u 

*8 


C' 

ON 

ON 


CM 

in 


Os 

CM 


r- 

CM 


NO 


CO 

in 


CZ) 

CZ) 


i-H 


C3 


00 


ON 


in 


C- 


CO 






cd 


CO 


^f 


p 


CM 


ON 






<D 

H 


in 


CM 


CM 


co 


CM 






£ 


in 


C- 


00 


NO 


of 








uo 


c- 


oo 


O 


NO 






Ch 


CM 


CM 


CM 








/*— \ 
N© 
ON 
ON 


di 

00 


^f 


CO 


co 


CO 


O 


u 


a\ 

CM 


CM 

CM 


CO 

CM 


00 

in 


CO 

CO 


<z> 

CZ) 




a 


o 


r- 


o 


Of 


c- 






cd 


CO 


CO 


oo 




of 






<D 

H 




^f 


NO 


ON 


O 






£ 


NO 




r- 


in 


in 








NO 


ON 


c- 


CM 


o 






Ch 


CM 


CM 


CM 


i—i 


i— i 


c 
















o 
















C/3 
















t . 


iTT 

ON 


aj 


r- 


O 


in 


r- 


00 


M 

c« 


00 


CM 


CO 


of 


p 


a ON 


C/3 


CM 


CM 


CO* 


in 


CO 


a 
















© 

r \ 




c 


o 


m 


Tf 


r- 


i-H 


u 








o 


cn 


ON 


NO 






<D 


NO 


o 


00 


O 


O 






£ 


NO 


r- 


NO 


of 


of 




















© 
















© 

pC 

© 




.2 




cd 


•« 

o 






CZ) 




£ 

o 




B 




r/s 












c 


> 


3 

X 

<D 

H 








"3 

a 


o 

M 


1 


o 

z 



o 

LO 



CD 

v~i 



Table 14 

Science Literacy Test 
Student Means 

Percent Correct on Multiple Choice Items 








in 




O 


cn 




o 

o 


a 


O 


O 

V 


ON 


oo 


O 

V 


o 














CN 


















NO 


CN 


CN 


CN 


m 




N 


ON 


in 


1 — * 


CN 


nr 


in 

©N 






in 

i 


o 


© 


NO 

i 


©n 




in 


o 


NO 




O 




< 


in 




i— J 


ON 


o 






CN 




O 


© 


ON 

1 






CN 




ON 


00 




Os 

©N 


P« 


NO 


o 

V 


OO 


in 


o 

V 


©N 
































O 


o 


nT 


m 


oo 


</> 


N 


in 




*— < 


in 


ON 


>■ 

IT) 

ON 




o 


tn 

i 


o 


o 


CN 

i 


ON 




CN 


CN 


ON 


oo 


OO 




< 


NO 


00 


*— J 


1— 2 


NO 






O 


NO 

i 


o 


cn 


nf 

i 










cn 




ON 


QO 

ON 


Pi 


o 


o 

V 


r- 


3 


r- 


ON 
































r- 


o 


in 


o 


NO 


</) 


N 


in 




cn 


VO 


<N 


irj 

ON 




CN 


nr 

i 


o 




O 


On 




oo 


NO 


NO 








< 


ON 


cn 


nr 


nT 


cn 






CN 


in 


o 


NO 


© 










m 


in 




r- 

ON 


P 


O 


o 

V 


in 


cn 


o 

V 


ON 














i-H 


















ON 


(N 


cn 


cn 


in 


J/5 


N 


NO 


00 


NO 


On 


n 


> 

IT) 

ON 


oi 


cn 

i 


O 

i 


© 


nf 

i 




ON 




o 


nr 


ON 


,— i 


in 




< 


o 


in 


ON 


o 


p 






cn 


nT 

1 


o 

1 


cn 


in 






tn 




ON 


ON 


o 


NO 

ON 


P 


r- 


o 

V 


r- 


o 


CN 


ON 














H 














# 




CN 


NO 


r- 


o 


ON 


!/> 


N 


cn 


in 


CN 


IN 


CN 


m 

ON 




o 


in 

i 


o 

i 






ON 




r- 


nr 


ON 


o 


cn 




< 


cn 


nr 


cn 


ON 


NO 






o 




o 

i 


in 


1 
















© 














o 














pC 














C-> 




d 










</> 




• i— H 

d 




d 


*- 

o 








c 

,o 




5 


>< 


c/5 






<4H 

"e3 

O 


Iowa 


d 

o 

2 


£ 

<L> 


cS 

X 

<u 

H 











OO 


nr 






d 


oo 


NO 


m 




r- 
















m ^ 


aJ 


CN 


oo 


o 


o 


ON 


y § 


o 


ON 






i— i 


O 

c/> CS 

V 


GO 




O 




cn 




d 


o 


CN 


in 


nT 


CN 




d 


r- 


CN 


in 


cn 


CN 




<D 

>— i 


ON 


O 




NO 


cn 




d 


nT 


NO 


NO 


nr 


cn 






o 


r- 


NO 


ON 


m 




d 


CN 


oo 


in 




m 






CN 










nf ^ 




in 


NO 


CN 


OO 


ON 


y £: 


o> 


ON 


OO 


O 




cn 


ON 


<y$ 


O 


o 




in 




C/5 ^ 
C/5 w 


d 


NO 


o 


oo 




in 




d 


l> 


m 


in 


NO 


in 




o> 

pH 




o 




OO 






E 


nT 


NO 


NO 


nT 


cn 






ON 


cn 


On 


cn 


CN 




d 


ON 


r- 


nr 


CN 


nT 






CN 










m ^ 
rj oo 

X ON 

on 




nT 


oo 


<N 


CN 


nr 


s.e 


00 

o 


o 


o 


O 

cn 


ON 

o 


C/5 C- 

CZ5 w 


d 


CN 


NO 


m 


o 


nr 




d 


i—i 


ON 


oo 


ON 


in 




o> 

pH 


o 






y—lt 


ni 




E 


in 


NO 


NO 


in 


nT 






o 


in 


, 


, 


CN 




d 


NO 


oo 


ON 


nT 


NO 






cn 










N ^ 


aj 


r- 


m 


NO 


m 


o 


y £ 


r- 


oo 


cn 


oo 


oo 


«8 S 

C/5 ^ 

C/5 


GO 


o 


o 






o 


d 


nT 


oo 


o 


m 


oo 




d 




r- 


nr 


nr 


o 




o> 


o 


CN 


o 


oo 






E 


m 


NO 


NO 


nr 


cn 






(N 


ON 


NO 


, 


oo 




d 


cn 


oo 


CN 


cn 


in 






cn 










y g 




cn 


nr 


o 


ON 


nT 


oi 


oo 


o 


CN 


; 


o 


ON 
C/5 ^ 

C/5 


t/3 


O 






CN 




d 


CN 


oo 


o 


cn 


in 




d 


in 


oo 


o 


cn 


oo 




o> 

pH 




ON 




J 


cn 




E 


nT 


in 


NO 


in 


nr 






nT 


nr 


00 


NO 


NO 




d 


CN 


ON 


00 


CN 


i—i 


c 




cn 


1—1 


CN 




CN 


o 














tfl ^ 














E in 

« ON 


oJ 


,—i 


cn 


i—i 


OO 


cn 


oo 


oo 


OO 


NO 




PON 


c/5 


o 


o 


o 


CN 


o 


E ^ 














o 


d 


nr 


<N 


On 


cn 


CN 


U 


d 




cn 


cn 


nr 


CN 




o> 

pH 


t> 


t> 




in 


CN 




E 


nr 


NO 


NO 


nr 


nr 
















© 














© 

u 




d 
• ^ 
rj 




d 


•a 

o 




VI 




E 

o 




s 


>- 


C/5 






Nm 

• fH 

”e3 

U 


Iowa 


d 

o 

2 


z 


S 

X 

o 

H 



"C-i 

& 



o 

(N 



in 

H 



Table 15 

Science Literacy Test 
Student Means 

Percent Correct on Open-Ended Questions 
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Table 16 
California 
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Student Means 

Percent of Time Spent by Class Activities in SS&C and Comparison Classes 
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Table 17 
California 

Student Questionnaire 
Student Means 

Frequency of Class Activities in SS&C and Comparison Classes 
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Table 18 
California 

Student Questionnaire 
Student Means 

Frequency of Inquiry Class Activities 
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Table 22 
California 

Teacher Questionnaire 
Individual Means 
Frequency of Class Activities 
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mean s.e. 
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mean s.e. 
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2.22 0.14 






<N 




fl 


oj 


co 




O 

.22 


C/5 


o 


o 


u If) 


so 






cd OS 


11 






a OS 


II 






S 






co 


o 


<L> 


<N 


oo 


U 


E 


co* 




b 














































cd 














b 


.2 






3 








a" 


cd 








t-H 

H 



T3 



<D 

> 

<D 

H— » 
C/5 

o 

s 

< 



in 

<D 

<D 

£ 

cd 

C/5 

<D 



2 

<D 

> 

<D 

05 



<D 

<u 



cd 

<D 

o 

CJ 

o 

2 

o 

X 

< 



co 

<u 

<D 

£ 

cd 

<D 

O 

a 

o 



C/5 

C/5 

<D 



(N 

»-T 

> 







CO 






cx 


<N 


oo 


o 








o 








© 








<N 




CO 


in 


Cfl 




CO 


(N 


► 




D 


©> 










OS 








OS 




CO 


so 


rH 


<: 


so 


© 






o 


©> 






CO 


(N 




Qh 


© 


i — i 


OS 








OS 








OS 








r-H 




00 


(N 


cw 


H— » 


so 








(N 


1— H 


If) 








OS 








OS 




Os 


© 


rH 


< 


OS 


CO 






© 


© 






CO 


r- 




Oh 


© 


© 


00 








OS 








OS 








r-H 




CO 


(N 


c« 


H— » 


so 


© 






<N 


<N 


1 f) 








OS 








OS 




On 


© 


r-H 


<3 


© 


in 








o 








<N 




Oh 


© 


© 


r- 








os 








os 








r-H 




or 


© 


C/5 


H— * 


© 


OO 


► 




<N 


(N 


IT) 








OS 








OS 








r-H 


<1 




SO 






© 


© 






so 


oo 




Oh 


© 


© 


so 








OS 








OS 








r-H 




(N 




Cfl 


H— » 




OS 


?► 




<N 


1— H 


ID 








OS 








OS 




oo 


ON 


r-H 


<1 


oo 


CO 






© 


© 


b 








.E 








<*■* 








u 








< 














cd 








a 






b 


.2 






3 


'"5 






cr 


cd 






0 


S-H 

H 



CO 

CD 



00 

(N 



CD 

rH 



Table 23 
California 

Teacher Questionnaire 
Individual Means 

Frequency of Inquiry Class Activities 
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Table 24 
California 

Teacher Questionnaire 
Individual Means 

Frequency of Traditional Class Activities 
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Table 25 
California 

Classroom Observations 
Percent Time Spent by Teaching Style 
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Table 26 
California 

Classroom Observations 

Frequency of Student Involvement and Type of Instruction 
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Table 27 
California 

Curriculum Verification 
Number of Weeks Spent on Science Topics 
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Table 28 
California 

Learning Environment Inventory 
Student Means 
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Table 31 - California 

Student Questionnaire - Student Means - Students’ Attitudes toward Science 
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Table 32 
California 

Student Questionnaire 
Student Means 

Percent of Students Who Have Participated in Science Activities Outside of Class 
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Table 33 - California: Student Questionnaire 

Percent of Students Who Have Participated in Science Activities Outside of Class (continued) 
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Table 34 
California 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Individual Class Activities 
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Table 35 
California 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Group Activities 
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Table 36 
California 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Whole Class Activities 
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Table 37 
California 

Achievement Scores 
Percent Correct on Achievement Tests 
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Table 38 
California 

Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score and Station 
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Table 39 
California 

Science Literacy Test 
Student Means 

Percent Correct on Multiple Choice Items 
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Table 40 
California 

Hands-On Full Investigation 
Student Means 

Percent Correct by Total Score and Subscores 








in 


IN 


IN 




CN 


O 






NO 


NO 


NO 


CN 


CD 


O 


o 

o 

o 

<N 


















CD 


(N 


ON 


ID 


O 


o 




N 


p 


p 


i— j 


CN 


O 


o 


> 




O 


O 


O 


, < 


, — ] 


o 


IT) 




i 






i 


1 




On 
















ON 




o 


N 


N 


N 


n 


o 




<| 


o 


NO 




p J 


NO 


o 






IN 


NO 




ON 

CN 

i 


NO 

i 


o 








NO 


O 


OO 




oo 




Oh 


ID 


CN 


CN 


oo 


NO 


p 


on 
















on 
















On 
































m 




CN 


CN 


N 


ID 


n- 


o 


Vi 


N 


NO 


p 


CN 


*— < 


p 


p 


>• 

id 

ON 




o 






o 


o 


o 

1 


ON 




CN 


O 


CN 


N 


CD 


p" 




<| 




o 


ON 


p 


CD 


CD 






ON 


o’ 


CN 




OO 


OO 








CN 


CN 






1 






N" 


o 


OO 


CN 


ID 


IN 




CX 


p 


o 


NO 


CD 


NO 


NO 


00 






* 










ON 
















ON 




































CD 


o 


*-H 


O 


ID 


CN 




N 


CD 


o 


p 


o 


p 


N" 


ID 

ON 




o 

1 


o 


o 


p 


O 

i 


© 

i 


ON 




oo 


o 


CD 


CD 


CD 


NO 




<] 


CN 


o 


CD 


OO 


CD 


ID 






UD 

i 


o 


OO 


o 

CN 

i 


OO 

1 


ID 

i 












ON 




P - 




Ph 


O 


o 


o 


CD 


o 


*— h 


n 

ON 








V 




V 




ON 
































m 




O 


1-H 


N" 


NO 


NO 


NO 


Vi 


N 


00 


NO 


00 


OO 


p 


p 


> 

ID 

ON 




CN 


CN 


CN 


o 


CD 




ON 




IN 


N 


N 


o 


CD 


IN 




< 


p 


NO 


NO 


ID 


CD 


NO 






00 


NO 


r— i 


CN 


CD 


NO 






CN 


CD 






CD 


r ” H 






ID 


N“ 


o 


ID 


IN 


O 




P 


CD 


p 


p 


CN 


NO 


o 


NO 














* 


ON 
















ON 
















▼-H 
















# 




CD 


N 


ID 


N" 


CN 


o 


<*} 


N 


on 


N 


OO 




Df 


o 


> 

ID 

ON 




o 


O 


o 




O 


o’ 


ON 




IN 


N 


N 


N 


CD 


o 




< 


NO 


NO 


NO 


NO 


CD 


o 








NO 


NO 


NO 


OO 


o 














































p 










ga 




GO 


<D 










P 


C/5 


P 


P 










O 


P 


O 


<D 

b 












.2 












43 


p 


*3 


p 

CO 


CO 








P 


"3 


p 


p 


OO 








O 


p 


o 


<D 


p 








U 


o 

U 


U 


2 


PQ 






<D 


Nm 


Nm 


Ph 


nw 






O 


o 


o 


o 


O 


o 






o 


1-H 








1-H 






CD 


<L> 


GO 


p 


Tj 


<D 






x> 


CO 


p 


O 


-O 






3 

o 

H 




<D 

O 

< 


O 


*& 

<D 

2 


3 

p 

Z 











o 


CD 


00 


o 












oS 


p 




O 


o 


ON 


p 


ID 






CO 


CD 


OO 


N 


On 


CN 


N 


u 


o 

o 

o 


NO 

II 
















CD 

CD 


CN 


P 


nean 


O 


N 


O 


N 


o 


N 






o 


NO 


O 


P 


o 


P 








ID 


NO 


o 




o 


NO 










ID 


NO 


ID 




ID 


NO 








a i 


00 


ID 


N 


o 


CD 


CD 








ID 


ID 


O N 


p 


p 


p 


Tf 






CO 






_ i 


ID 


N- 


OO 


U 


ON 

ON 

ON 


P" 

II 










CN 






CD 

CD 




P 


nean 


CN 


O 


ID 


O 


O 


CD 








O 


IN 


o 


O 


p 










o 


OO 


ID 


ID 


00 










IN 


oo 


NO 


C — 


IN 


ID 








0) 


NO 


ON 


CN 




N 


N* 








O 


oo 


O 


OO 


CD 


p 


CD 


z^\ 




CO 


CD 


N 


ID 


ID 


ID 


O 


u 


00 

ON 

ON 


NO 

II 
















CD 

CD 




P 


nean 


CN 


o 


N 


o 


CD 








IN 


p 




o 


p 


p 








NO 


o 




o 


00 


i— ) 










ID 


NO 


ID 


ID 


ID 


NO 








oJ 


ID 


CD 


ON 


N 


O 


ID 








OO 


CD 


p 


CN 


o 


p 


CN 






CO 


CN 


CD 


ID 


ID 


o 


N 


u 

«8 


F 

ON 

ON 


NO 

II 
















i/3 

C/3 




P 


nean 


IN 


N 


O 


CD 


o 


CD 






I-* 


P 


ID 


p 


p 


p 








O 


NO 


N 


CD 


o’ 


CD 










On 


ON 


00 


00 


o 


00 




























oJ 


OO 


N 


N 


ON 


00 










C\ 


p 


O 


p 


o 


NO 




/— N 




CO 


IN 


NO 




ID 


N 


00 


U 


NO 

ON 

ON 


NO 

II 
















C/3 

C/3 




P 


nean 


N 


N 


o 


O 


o 


N 






NO 


NO 


p 


ID 


p 


NO 








CD 


NO 


CN 


N 


ID 


NO 










IN 


I — 


NO 


OO 


C — 


NO 










ID 


NO 


N 


N 






P 

o 

CO 






s.e 


NO 

On 


P 

CD 


p 

CD 


P 

CD 


ID 

O 


NO 

00 


’E 


ID 


NO 










’—i 






C4 


ON 


II 
















Qri ON 
















a 




P 


nean 


O 


o 


CD 


CD 


N 


N 


o 






O 


o 


OO 


00 


NO 


NO 


U 






CN 


o 


ID 


O 


NO 


NO 










NO 


NO 




IN 


NO 


NO 


















+-► 




















P 














cn 




m 


<D 














P 


cn 


P 


g 














O 


P 


O 


i-H 
















.2 




2h 
















NP 


'O 


p 

C/5 


C/5 












P 


^3 


P 


P 


oo 












o 


p 


o 


<D 


P 












U 


o 

U 

o 


O 


s 


OQ 










d> 

Uh 

O 


o 


<4H 

o 


l+H 

O 


o 










o 


1-H 








1-H 










CD 


<D 


GO 

in 




T3 

O 


<D 

x> 










3 

o 

H 




(D 

O 

< 


o 


*5 

<D 

s 





' T* 

o 

C\i 



5 ? 



O 

C4 



Table 41 
California 

Written Full Investigation 
Student Means 

Percent Correct by Total Score and Subscores 
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Table 42 
Iowa 

Student Questionnaire 
Student Means 

Percent of Time Spent by Class Activities in SS&C and Comparison Classes 
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Table 43 
Iowa 

Student Questionnaire 
Student Means 

Frequency of Class Activities in SS&C and Comparison Classes 
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Individual Means 
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Classroom Observations 

Frequency of Student Involvement and Type of Instruction 
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Curriculum Verification 
Number of Weeks Spent on Science Topics 



0 

ERsLC 



u ° 

© 

& a 
c n w 



c 

o 

W5 

u IT) 
C$ Os 
©on 

s d. 

o 

u 



'a 

u 

c$ 

© 

c 

c$ 

C/5 



T3 

<D 

4— * 

O 

£ 

c3 

GO 

O 

on 



O 



On 

II 



T3 
<D 
4— » 

o 

£ 

c3 

GO 

<D 

GO 

J 



N° 

o x 



GO 



T3 

<D 

't— • 

O 

£ 

a3 

GO 

<D 

GO 



o 



GO 

M 

<D 

I 



T3 

<D 

4 -* 

o 

,<D 

to 

CO 

CO 

<D 

CO 



o 



N° 



CO 

M 

<u 

I 



T3 

<U 

4 -* 

o 

iS 

C3 

GO 

<D 

GO 

J 

73 



o x 



GO 

<D 

I 



T3 

<U 

4—* 

o 

,<D 

to 

cd 

GO 

<D 

GO 



o 



oo 



GO 

M 

<u 

I 



NO 


Tf 


oo 


CN 


o 


o 


o 


IT) 






CN 


o 


o 


o 


o 


C\ 




CN 


o 


o 


o 


cn 




CN 


CN 








o 


o 


© 


O 


o 


o 


o 


,—5 


t> 


o 


OO 


o 


o 


o 


















o 


CN 




o 


o 


o 


in 


*— ' 


*— ' 


CN 


o 


o 


o 


oo 


oo 


NO 


t> 


o 


o 


o 


CN 


cn 




CN 








m 


o 


cn 


© 


o 


o 


o 


O 




cn 


o 


o 


o 


o 
















o 


o 


N" 


CN 




o 


o 


cn 


l> 


*— ' 


cn 


in 


o 


o 


4 




cK 


NO 


in 


o 


o 


cn 


CN 




(N 








cn 


O 


cn 


in 


o 


o 


o 


1— H 


© 


cn 


ON 


cn 


o 


o 


















NO 


NO 


r- 


o 


o 


o 


^3- 


o 


o 




o 


o 


o 


ON 


cn 


CN 


NO 


o 


o 


o 


CN 


(N 


(N 


<N 








O 


in 


in 


O 


o 


o 


o 


O 


t> 




ON 


o 


o 


o 


















ON 












cn 


OO 


o 


ON 


o 


o 


o 


in 


in 


o 


00 


o 


o 


o 


o 




o 


l> 


o 


o 


o 




cn 




in 








o 


o 


o 


o 


o 


o 


o 




CN 


o 


<N 


o 


o 


o 








CN 








NO 


NO 


cn 


NO 


o 


o 


o 


00 


00 




00 


o 


o 


o 


CN 


CN 


1-4 


CN 


o 


o 


o 




N" 


in 










O 


O 


o 


O 


o 


o 


o 


oi 


© 


NO 


CN 


o 


o 


o 




cn 


cn 














GO 














<D 














O 














c 














2 

*3 






& 








oo 






<D 




E? 




<D 

O 

cd 

c n 




"H 

(D 

s 


*3 

o 

cn 

<* 


GO 

o 

<D 

g 


4—* 
. CO 


0X5 


<« 


GO 

O 


1 


<D 

o 


M 


s 

<u 

43 

u 


o 

5 


1 

W 


’ 7 

X? 

Oh 


s 

u 

s 


c 

'U 

*3 

00 


<u 

% 



on 



o> 

c\j 

c\j 



23 0 



Table 54 
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Learning Environment Inventory 
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Table 56 
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Table 57 - Iowa 

Student Questionnaire - Student Means - Students’ Attitudes toward Science 
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Table 58 
Iowa 

Student Questionnaire 
Student Means 

Percent of Students Who Have Participated in Science Activities Outside of Class 
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Table 59 - Iowa: Student Questionnaire 

Percent of Students Who Have Participated in Science Activities Outside of Class (continued) 
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Teacher Questionnaire 
Individual Means 

Percent Time Spent in Individual Class Activities 
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Teacher Questionnaire 
Individual Means 

Percent Time Spent in Group Activities 
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Teacher Questionnaire 
Individual Means 

Percent Time Spent in Whole Class Activities 
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Achievement Scores 
Percent Correct on Achievement Tests 
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Classroom Observations 
Percent Time Spent by Teaching Style 



O 

ERIC 



o© o 

co ci 
co 



o\ 

on 
CO C 
CO 



00 
» O' 
on 
CO C. 
CO 



U gr 

o8 Os 

co C- 
co 



s® 

o x 



m 



cr 



in 



cr 

<D 

* 



VO 



cr 

© 

* 



U g 

ON 
CO Cl 

C/5 



fi 

o 

.£* /*■*> 

U IT) 
« Cs 

ac\ 
E C 

o 

U 



C/5 



cr 

© 



VO 



cr 

<D 

* 



cr 

© 

* 



oo 



m 



VO 



r- 

vq 

vd 



in 



o 

p 

<N 

<N 



t> 

O 

(N 



l> 

in 

cn 



r- 



(N 

vd 



© 

o 

cd 

© 

H 

C« 



s 

a 

05 O 



<N 



VO 



in 



m 



vo 



vo 



(N 



ov 



C/J 

© 

H 

a 

© 



Cd 

+■> 

C/5 



C/5 


















O 

V 



o 

m 



vo 



o 

V 



(N 

Tj- 

(N 



oo 

<N 

cn 



O 

V 



r- 

oo 



oo 

VO 



o 

V* 



p 

a 

^r 



m 

<N 

p 

a 

6v 

^r 

OV 



in 

o 



in 

oo 

OV 

vo 



o 

V 



m 

co 



c /5 

© 

j 2 

a 

> 

a 

o 



o 

N 



© 

43 

0 
a 
© 

H 

<« 

1 TD 
© 

g & 

3 = 

S © 

GO O 



T 3 

© 



c 

0) 

U 

i 

i 

CO 



© 

> 

a 

* 

C/5 

• t-H 

CS 



C/5 

© 



TD 

< 



cd 

l-t 

JS 



!>* 

r- 

C\i 



278 



Classroom Observations 

Frequency of Student Involvement and Type of Instruction 
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Curriculum Verification 
Number of Weeks Spent on Science Topics 
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Student Means 
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Table 85 - Montana: Student Questionnaire 

Percent of Students Who Have Participated in Science Activities Outside of Class (continued) 




*r> ~ 

og o 

go e. 

c/5 w 



CN 

in 



5 g 
§\ 
C/5 C 
C/5 w 



ro 

) go 

©8 ON 

co C 
co 



CM 

nT 



U ■ 

00 ON 

co : J 

cz> 



> S 

q 8 o\ 

co : ■ 

CO 



c 

© 

^5ft 

*E 

a 

E 

o 

U 



£ 

. 5 ; 

© 

< 



L © 

C cd 

<d 

s 



in 

cn 



NO 

cn 

CM 



<U 



nT 

nT 



O, 

cn 

cn 



c- 

CM 

cn 



MD 

nT 



MD 

OS 

CM 



c- 

cn 

CM 

CM 



O', 

o 



r- 

NO 



o 



CM 

in 

cn 



nT 

in 

CM 

CM 



NO 

O 



r- 

CM 

ON 

CM 



r- 

CM 

nT 

CM 



m 

On 

CM 



c- 

ON 

CM 



00 

CM 



ON 

cn 



o 

NO 

NO 

cn 



in 

o 



ON 

in 

CM 

cn 



o 

cn 



NO 

© 

NO 

CM 



cn 

in 



NO 

cn 



ON 

NO 



oo 

ON 

cn 

cn 



o 

cn 



O 

NO 

nT 

cn 



CD 

O 

© 

<d 

o 

C/3 

© 

O 

Cd 

C/3 

o 

O 

x> 

cd 

Pi 



© 

<L> 

s 

c 

CD 

& 

X 

<d 

CD 

o 

© 

JD 

o 

cft 

cd 

O 

Q 



in 

NO 



OO 

OO 

cn 



m 

NO 



On 

CM 

in 

cn 



NO 

nf 



NO 

ON 

CM 



cd 

J-H 

iS 

JD 

13 



S 

I 



CM 

nT 

00 

cn 



ON 

ON 



00 

ON 

cn 



o 

oo 

cn 

cn 



o 

CM 



O 

CM 



ON 

nT 

CM 



O 

OO 

NO 



cn 



CM 

CM 



cd 



NO 

in 



NO 

00 

O 

cn 



nr 

ON 



r- 

in 

nr 



CD 

O 

a 

o 

'o 

C/3 



in 

cn 



in 

in 

cn 



<L> 

© 



00 

cd 

B 

CD 

O 

© 

*o 

cft 

cd 



Pi 



Oft 

<U 

H 



Cd 

© 



cd 

CO 







r- 


in 


ON 






in 




P 


00 




NO 




o 


NO 


© 
















© 
















o 
















CM 




r- 


cn 


O 




nr 


in 




N 


j 


nT 


nr 




nr 


nT 


> 




o 


1—4 


© 




CM 


O 


m 






i 






1 


i 


ON 
















ON 




o 


nT 


o 




r- 


NO 




< 


cn 


nr 


o 




in 


CM 






© 


NO 

1 


CM 




cn 

i 


CM 

i 






ON 


in 


o 




CM 


O 




P 


ON 


nr 


c- 




in 


NO 


ON 
















ON 
















ON 






















NO 


ON 




nr 


cn 


Cft 


N 


© 




cn 




NO 


in 


> 

in 




© 


© 


o 




i 


o 


ON 
















ON 




iH 


CM 






NO 


nT 




<] 


o 


NO 


o 




CM 


l> 






o 


cn 


CM 

i 






CM 

i 






ON 




OO 




ON 


NO 




P 


CM 


1— < 


o 




O 


nr 


OO 
















ON 
















ON 




















in 


c- 


c- 




1-H 


nr 


Cft 


N 


o 


cn 


t> 






t> 


> 




1—4 


1-H 


1—4 




1—4 


o 


in 




1 


• 


1 




1 


1 


ON 
















ON 




NO 


r- 


nr 




1-H 


in 




< 


in 


CM 


in 




OO 










NO 


00 




CM 


cn 






oo 


nr 


o 




ON 






P 


CM 


ON 


oo 




o 


CM 


h 
















ON 
















ON 
















?— < 




OO 


oo 


in 




o 


1—4 


6ft 


N 


o 


o 


CM 






— ; 


> 




i— i 


o 


o 




1-4 


1-H 


m 




1 








1 


1 


ON 
















ON 






NO 


nr 




CM 


ON 




<1 


r- 


nr 


in 




o 


NO 








© 






cn 


NO 






oo 


oo 


( 




NO 


ON 




P 


ON 


cn 


ON 




cn 


O 


NO 
















ON 
















ON 
















T-H 




cn 


oo 






CM 


^H 


Cft 


N 


o 


oo 


i— J 




ON 


r> 


> 




© 


© 


o 




o 


T—I 


in 




1 


1 


1 








ON 
















ON 




NO 


nr 


CM 




oo 


CM 




<] 


o 


in 


NO 




in 


O 






© 


nT 


O 




CM 


o 






1 


1 


1 














CD 
















O 








CD 








c 


c 






c 








CD 


CD 






•fl 








o 


s 


cd 




Cd 








C/3 


‘C 






gp 










CD 


o 




cd 


* 
• pN 




CD 

P 


© 

o 


& 


P 

CD 




B 

CD 


> 




o 




CD 




Id 






o 


cd 


13 




O 


*•*3 




C/3 


CD 




P 


C 


© 

< 




JD 

13 

+-> 


C/3 

o 

Q 


O 

c 

CD 

• 


1 

c 


O 


CD 

• *-H 

O 

Cft 






cd 




O 

eft 


3 

2 


(D 

o 


cd 






2 


n3 


cd 


a 


~o 








cd 




(D 


cd 






© 

CQ 


Pi 


O 

Q 


1 


• p-H 

O 

Cft 


V 

Pi 



CD 

C\J 



293 



eft 

QJ 



CJ 

◄ 

eft 

eft 

« 



Qj 

a 
a 
© 



« 5 

_ a> ^ 
ve s ,g 5 r 

oo h *- « .5 

» 2 2 -os ■§ 

£ gx 3 w 

5 1 ? 1 -s 

a> .£ 

X! x> a 
a ta » 
* « a 



a> 

H 



C /5 

a> 



a 

eu> 

o 

0 > 

eu 




(N 

ON 



LO 

03 

CVi 



O 

ERLC 

iamaffamiaaa 



296 



Teacher Questionnaire 
Individual Means 

Percent Time Spent in Group Activities 
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Teacher Questionnaire 
Individual Means 

Percent Time Spent in Whole Class Activities 
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Achievement Scores 
Percent Correct on Achievement Tests 
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Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score and Station 
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Table 103 
New York 

Classroom Observations 
Percent Time Spent by Teaching Style 
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Table 104 
New York 

Classroom Observations 

Frequency of Student Involvement and Type of Instruction 
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Table 105 
New York 

Curriculum Verification 
Number of Weeks Spent on Science Topics 
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Table 106 
New York 

Learning Environment Inventory 
Student Means 
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Table 109 - New York 

Student Questionnaire - Student Means - Students’ Attitudes toward Science 
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Table 110 
New York 

Student Questionnaire 
Student Means 

Percent of Students Who Have Participated in Science Activities Outside of Class 
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Table 1 1 1 - New York: Student Questionnaire 

Percent of Students Who Have Participated in Science Activities Outside of Class (continued) 
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Table 112 
New York 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Individual Class Activities 
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Table 113 
New York 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Group Activities 
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Table 114 
New York 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Whole Class Activities 
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Table 115 
New York 
Achievement Scores 
Percent Correct on Achievement Tests 
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Table 116 
New York 

Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score and Station 
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Hands-On Full Investigation 
Student Means 

Percent Correct by Total Score and Subscores 






















CL. 


O 


p 


o 


O 


o 


o 


o 

o 

o 




V 




V 


V 


V 


V 




NO 




CD 




o 


<N 


c« 


N 


CD 


NO 


to 


p 


© 


p 


id 

ON 




ID 


, “ H 


NO 


CD 




id 


ON 

^pM^ 




CD 


O 


O 


O 


o 


r- 




<| 


ON 


o 


© 


o 


o 


NO 






X 


oo 


o 


ID 


o 


NO 








<N 




ID 










r- 


ON 


<N 


O 


CD 






cl. 


o 


i-H 


T— H 


CD 


<N 


o 


ON 

ON 














V 


ON 




































ON 


o 


ID 




o 


<N 


c« 


N 


p 


CD 


ip 


O 


<N 


p 






,___| 


1-H 




,_l 


T—{ 


ID 


IT) 
















ON 
















ON 

^pM^ 




r- 


CD 


o 


o 


CD 


r- 




< 


cnj 


CD 


o 


© 


CD 


NO 






NO 


, — ' 


o 


© 


CD 


NO 






CD 


X 




CD 


<N 








t> 


OO 




r^ 


O 


NO 




CL 




CD 


(N 




O 


ID 


oo 












* 




ON 
















ON 
















H 




















<N 




NO 


<N 


o 


00 


CO 


N 




00 


(N 




o 


ID 


> 

ID 

ON 




© 


o 




o 


o 


O 


Os 

^pM^ 






o 


O 


o 


o 






< 


NO 


o 


O 


o 


o 


NO 






ON 


o 


o 


ID 


© 


NO 








<N 


(N 






i 






O 


CD 


Df 


OO 


ID 


OO 




CL 


ID 




CD 


00 


OO 


o 


r- 
















ON 
















Os 




































OO 


ON 


NO 


ID 


ON 


r- 


CO 


N 


NO 


p 


ON 




r— I 




> 

ID 

ON 




o 


o 


o 


o 


o 




ON 




NO 


r- 


o 


o 


r- 






< 




NO 


o 


o 


NO 








CD 


Dt- 


ID 


ID 


NO 








1—1 




1—1 






<N 






NO 


x 


o 


Os 


r- 


Tf 




cl 


ON 


(N 


CD 


NO 


y — < 


ON 


NO 
















ON 
















ON 
















wH 




















UD 


r- 


CD 


O 


r- 


r- 


CO 


N 


O 


’ — ' 


© 


Dl" 


CD 


o 


1> 

ID 

ON 




© 




1 


© 


i 


o 


ON 

^pM^ 






r- 


CD 


CD 


r- 

NO 


<N 




<] 


o 


NO 


CD 


CD 


(N 








CD 


OO 

i 


CD 


NO 

(N 

i 


(N 






























G 










CO 




CO 


<D 










a 


CO 


g 


g 












G 


.2 


| 










X 


.2 


X 










t3 


x 


*3 


CO 


CO 








G 


•3 


g 


CCj 


OX) 








o 


g 


o 


<D 


s 








U 


o 

U 


U 


s 


CQ 






£ 


Nm 


Nm 


Nm 


4-1 






o 


o 


o 


o 


O 


o 






o 


Vh 




■*—» 




t-l 






CD 


<L> 


CO 


c 


T3 


0) 






X) 


C/3 


p 


O 


X 






B 

o 

H 


2 


<L> 

o 

< 




■5 

<D 

S 


2 









ci 


o 


o 


o 


O 


© 


© 








o 


o 


o 


o 


© 


© 


ID 


/- _ v 




CO 


o 


CD 


CD 


CD 


© 


© 


u 

4 


o 

o 

o 


II 
















C/3 

C/3 


<s 

'w' 


c 


nean 


o 


o 


o 


o 


© 


© 






p 


o 


o 


o 


© 


© 








C> 


o 


ID 


o 


CD 


© 










oo 


X 


o 


o 


© 


© 




















1 










r- 


t> 


o 


o 


r- 


© 








oi 


p 


X 


p 


p 


X 


© 


Tf 






CO 


oo 


NO 


ID 


ID 


NO 


© 


U 

«8 


ON 

ON 

ON 


CD 

II 






<M 


<N 


<N 






C/3 

C/3 


wH 


c 


nean 


CD 


CD 


O 


o 


CD 


© 






CD 


CD 


O 


p 


CD 


© 








P 


CD 


ID 


ID 


CD 


CD 










OO 


r- 


r- 


o 


OO 


© 




























6 


CD 


r- 


00 


o 


© 


X 








ON 


as 


ID 


p 


p 


p 


to 


/- _ v 




CO 


o 




TJ- 


ID 


CD 


OO 


u 

«8 


00 

ON 

OS 


ID 

II 
















C/3 

C/3 




c 


nean 


CD 


o 


O 


o 


© 


r- 






r- 


o 


O 


o 


© 


X 










CN 


ID 


o 


© 


NO 










ID 


ID 


ID 


X 


X 


X 








ci 


00 


r- 


CD 


r^ 


CD 










ID 


X 


p 


00 


p 


i— « 


<S 






CO 


K 


NO 




oo 


CD 


X 


u 

«8 


P 

ON 

ON 


CD 

II 










CN 


CD 




03 

i/3 


rH 

'w' 


p 


nean 


<N 


r- 


O 


o 


r-* 


00 






<N 


X 


o 


o 


X 


r- 








OO 


NO 


o 


CD 


NO 












ID 




ID 


ID 


X 


r- 








oi 






r- 


ID 


r- 


© 








’—i 


o 


X 


O 


X 


Df 


rH 






C/5 


H 




NO 


© 


NO 


ON 


u 

«8 


NO 

ON 

ON 


CD 

II 




<N 


<N 




CD 




CN 


C/3 

C/3 


wH 


G 


nean 




r- 


r- 


CD 


CD 


X 






’-H 


X 


X 


CD 


CD 


ip 








NO 


NO 


NO 


OO 


CD 


ID 












X 




ID 


CD 


ID 








ci 

CO 


<N 


^r 


CN 


OO 


© 


X 


B 

o 

CO 






OO 




NO 


Ip 

Tf 


© 

© 


OO 


*C 


P 


ID 










’—1 


i 






ON 


II 
















CL ON 
















a 




G 


nean 


r- 


o 


O 


o 


© 


CD 


o 






o 


o 


O 


© 


© 


CD 


U 






ID 


CN 


ID 


ID 


© 


CD 












CD 


CD 




X 


ID 


















+-» 




















G 














CO 




CO 


<U 














G 


CO 


G 


E 














O 


G 


.2 


— H 
| 














X 


.2 
















'S 




X 


CO 


CO 












G 


•3 


G 


cd 


OX) 












O 


G 


O 


<u 


G 












U 


O 

U 


U 


s 


PQ 










<U 


4-i 


4-i 


4-( 


4h 










o 


O 


o 


O 


O 


O 










o 


Vh 




-M 




t- 










C/D 


d> 


CO 


c 


X 


<u 










X 


CO 


p 


O 


X 










3 

(2 


2 


<D 

O 

O 

< 




X 

<u 


2 



(N 



05 

LO 

05 



360 



Table 119 
New York 

Written Full Investigation 
Student Means 

Percent Correct by Total Score and Subscores 
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Table 120 
Texas 

Student Questionnaire 
Student Means 

Percent of Time Spent by Class Activities in SS&C and Comparison Classes 
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Table 121 
Texas 

Student Questionnaire 
Student Means 

Frequency of Class Activities in SS&C and Comparison Classes 
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Table 122 
Texas 

Student Questionnaire 
Student Means 

Frequency of Inquiry Class Activities 
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Table 123 
Texas 

Student Questionnaire 
Student Means 

Frequency of Group Class Activities 
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Table 124 

Texas 

Student Questionnaire 
Student Means 

Frequency of Traditional Class Activities 
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Table 125 
Texas 

Student Questionnaire 
Student Means 

Frequency of Other Class Activities 










O 


ON 


<N 




o 








o 


i—i 




1—1 


m 






cn 


o 


O 




o 


u 


O 

o 

o 


O 

II 

G 












m 

Xfl 


w 


mean 


Tf 

ON 


NO 

Tf 




m 

Tf 










ni 














OO 


o 




ON 








D 


O 


1—1 




O 


Tf 


/*“ N 


<N 


cn 


O 


o 




o 


U 


ON 

ON 

ON 


Tf 

II 

G 












<Z) 

tZ) 




G 

cd 

D 


ON 

r> 


o 

ON 




OO 

NO 








E 
















O 


r- 


o 




OO 








o 


<— » 




o 




y— N 


ON 


cn 


o 


o 




o 


U 


00 

ON 

ON 


in 

II 

G 












tZ) 

m 




mean 


r- 

ON 


in 

<N 




o 

r- 










cn 












O 


r- 


oo 




ON 








o 


o 




O 


fS 




(N 


on 


o 


o 




o 


U 


F 

ON 

ON 


O 

(N 

II 

G 












m 

m 




mean 


r- 


ON 

Tf 




cn 

oo 








(N 


(N 












O 


NO 


OO 




00 








O 


o 




o 


t-H 


/-— s 


m 


on 


O 


o 




o 


U 


NO 

ON 


CN 














ON 


II 

G 












m 

m 




mean 


Tf 

Tf 


ON 




m 

ON 








(N 


(N 












qj 


in 


r- 






c 






o 


o 




o 


o 

5fl 




m 


on 


o 


o 




o 


*C 


iTT 














Cd 


ON 


(N 












& ON 


II 












a 

o 




II 

G 


mean 


o 


Tf 

in 




oo 

Tf 


U 






(N 


cn 
















on 
















O 
















CD 
















-g 
















’> 














o 
















£ 


o 






g 








o 


on" 

D 


on 

on 

cd 












TO 
















O 




Id 




'•*3 








G 


In 






< 








IS 

rt 


on" 


£ 












G 


s 


D 


G 










O 




TO 


o 










u 


s 

43 


1 


VG 

o 










13 


S 


o 


2 










-4-* 

H 


cd 

£ 


o 

O 


-4-J 

cn 

.£ 











r- 


m 




ON 










Tf 


in 




NO 






o 
















o 
















o 
















<s 




m 


Tf 




O 






cn 






NO 




Tf 










o 


O 




o 






in 




i 


i 




1 






ON 
















ON 




r- 


OO 




in 








<] 


o 


o 




o 


& 








o 


o 




o 


& 








<N 






r- 


CD 

> 






D, 


O 


o 

V 




o 


CD 




ON 










'*— • 




ON 










cn 




ON 












O 

c 








in 


r- 




Tf 


G 




cn 




Tf 


Tf 




OO 


< 








(N 


in 




1-^ 


II 




in 




1 


i 








ON 












in 




ON 




(N 


Tf 




o 








<] 


(N 


NO 




CN 


CD 








O 


O 




o 


CD 








1 


* 






£ 
















cd 
















cn 

CD 

G 








in 






m 






Qh 


NO 


o 




o 




00 












*— i 




ON 












+-* 




ON 












13 








in 


NO 




Tf 


CD 




cn 


S-* 


Tf 


in 






> 




> 




o 


(N 




CN 


CD 




m 




i 


i 






cn 




ON 












II 




ON 




^f 


O 




CN 


Tf 


«n 

+-» 




< 


o 


m 




CN 








o 


o 




O 


M 


cn 






1 


* 






CD 

CD 


D 














H 














£ 


cd 






r-* 


ON 






cd 


CD 




a. 


o 


in 




o 


CD 


« P—1 
*-* 


l> 










V 


o 


cn 


ON 










G 


+3 


ON 












o 


cd 






m 


Tf 




CN 


G 


m 




•4-* 


00 


in 




cn 


O 




> 






o 




cn 


4=> 

< 




in 

ON 










II 

m 




ON 




in 


NO 




in 






< 


o 


O 

o 




cn 

o 


4*f 










1 






(D 
















D 

£ 






























cd 






cx 


o 


o 




o 


D 




NO 




V 


V* 




V 


O 




ON 






G 




ON 












O 








o 


oo 




CN 


§ 




cn 


’t-* 


Tf 


o 




cn 










in 


Tf 




Tf 






in 












on 




ON 












cn 

D 




ON 




m 


cn 




in 








< 


Tf 


Tf 




Tf 


II 








o 


O 




o 


CN 
















j-T 

D 

> 

D 

z 

11 








nr 


cn 

O 

D 

12 












D 


1-1 
















O 


cn 

cn 

a 












o 


cn" 

D 








• mm 






"O 














D 




13 








33 




G 




i-t 








D 

< 




IS 


cn" 


£ 












E 

o 


M 


D 

TO 


G 

o 










U 


IG 

43 


1 


• ^ 
CD 










H 




o 

o 

O 


2 

-4-J 

cn 

^G 



nr 

j> 

co 



373 



Table 126 
Texas 

Teacher Questionnaire 
Individual Means 
Frequency of Class Activities 



O 

ERIC 









D 


cN 


O 








(N 




1/3 






c/3 


© 


© 


u 


O 

O 

© 


m 

II 








05 

05 


w 


G 


mean 


r- 

NO 


00 

l> 








cn 


CN 








D 


00 


CN 








*— < 


m 


Tf 






C/3 


© 


© 


u 

«8 


P 

ON 

ON 


’'3' 

II 








05 

05 


S— ✓ 


C 


mean 


© 

to 


CN 








cn 


CN 








D 


m 












(N 


m 


\ 




in 


© 


© 


U 

*8 


00 

C\ 

ON 


II 








05 




C 


mean 


ON 


00 


05 






NO 


© 








cn 


cn 








CD 


00 


o 








o 




cn 






c n 


o 


© 


W 

*8 


P 

ON 

ON 


CO 

II 








05 

05 


▼™C 


C 


mean 


oo 

© 


00 

l> 










CN 








D 


cn 


NO 








CN 


NO 


vH 






in 


© 


© 


U 

«8 


(o 

ON 

ON 


tn 

II 








!/) 

CC 


vH 

'w' 


C 


mean 


© 

cn 


cn 








Tf - 


cn 










Tf 


NO 


a 






D 




NO 


o 

C/3 






in 


© 


© 


*c 


oT 


^3" 








C3 


ON 


ll 








a on 


II 








E 




C 


mean 


NO 


oo 


o 






to 


oo 


U 






rn 


CN 


£ 












.> 
























D 


































d 












c 










P 


.2 










3 


'S 










cr 


c3 










c 


i-H 

H 



£ 

T3 



O 

> 

CD 

> 

m 

O 

E 



ij 

I 

m 

CD 

E 



03 

CD 

> 

CD 

C/3 



M 

CD 

CD 

£ 

cd 

CD 

O 

C 

o 

'S 

o 

< 



<D 

D 



d 

D 

O 

G 

O 



m 

m 

D 



<N 



D 

> 

D 

£ 







VO 


1-H 




a 


oo 


ON 


© 








© 








© 








N 




ON 


CN 


xA 




1 


’“J 


> 




© 


© 


in 






1 


ON 








ON 




© 


© 


*H 


<3 










© 


© 






o 


to 




a 


ON 


in 


ON 








ON 








ON 












cn 


cn 


xA 




W — H 


VO 


> 




© 


© 


in 




1 


1 


ON 








ON 




NO 


© 


*H 


< 


© 


nf- 






© 


© 






^3" 






Oh 


oo 


r- 


00 








ON 








ON 








vH 






© 


xA 




CN 


cn 


> 




© 


© 


in 








ON 








ON 




cn 




v—i 


<] 




CN 






© 


© 






NO 






Cu 


cn 


ON 


r- 








ON 








ON 








vH 




© 


CN 


xA 




© 




> 




*-h 


© 


in 






1 


ON 








ON 




<N 


© 


*H 


< 


in 








© 


© 








© 




a, 






NO 








ON 








ON 








tH 




ON 




xA 




NO 


CN 


> 




*— H 


© 


in 








ON 








ON 




ON 


VO 




< 




CN 






© 


© 


£ 
























D 








< 






( 








d 








C 








.2 






3 


© 






a" 


g 






C 


J-H 

H 



CD 

r- 

CD 



CN 

m 



375 



Table 127 
Texas 

Teacher Questionnaire 
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Frequency of Inquiry Class Activities 
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Table 128 

Texas 

Teacher Questionnaire 
Individual Means 

Frequency of Traditional Class Activities 
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Table 129 

Texas 

Classroom Observations 
Percent Time Spent by Teaching Style 
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Table 131 

Texas 

Curriculum Verification 
Number of Weeks Spent on Science Topics 
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Table 132 
Texas 

Learning Environment Inventory 
Student Means 
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Table 137 - Texas: Student Questionnaire 

Percent of Students Who Have Participated in Science Activities Outside of Class (continued) 
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Table 138 
Texas 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Individual Class Activities 
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Table 139 
Texas 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Group Activities 
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Table 140 
Texas 

Teacher Questionnaire 
Individual Means 

Percent Time Spent in Whole Class Activities 
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Table 141 

Texas 

Achievement Scores 
Percent Correct on Achievement Tests 
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Table 142 

Texas 

Lab Skills Performance Test 
Student Means 

Percent Correct by Total Score and Station 
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Table 145 
Texas 

Written Full Investigation 
Student Means 

Percent Correct by Total Score and Subscores 
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Table 146 

Student Outcome Effect Sizes 



00-56 


I 0£'0 


<N 

VO 

d 


1.24 | 


1 on 


cn 

Tf 

d 

■ 


o 

d 




00 


VO 


r- 


o 


^r 






o 


CO 


00 


1 


o 


o 


66-56 


© 


d 


d 


d 

i 


d 


d 




m 


<N 


m 


<N 


cn 






Tf 


Tf 


(N 


00 




© 


86-56 


© 


d 


" 


d 


d 

i 


d 




m 


os 


<N 


^r 


r- 


r- 




CO 


00 




r- 


(N 


p 


Z,6"56 


© 


d 


d 


d 


d 


d 

i 




(N 


VO 


00 






m 




cn 


p 


p 


r- 


1 


p 


96-56 


d 


d 




d 


d 


d 

i 




<N 


o 


l 




tT 


OV 




p 


1 


oo 


< 


p 


p 


00-56 






d 


d 

i 


d 

i 


d 

i 




o 


VO 


_ 


<N 


o 


00 




(N 


Tf 


p 


<N 


p 


p 


66-56 




d 


d 


d 


d 

i 


d 

i 




<N 


VO 


00 


<N 


<N 


Os 




<N 


Tf 


p 


Tf 




Tf 


86-56 




d 


d 


d 


d 

i 


d 

i 




VO 


00 




m 




Os 




<N 




p 




p 


p 


Z.6-56 




d 


d 


d 

i 


d 


d 

i 




o 


r- 


m 


VO 


(N 


r- 




— 1 


p 


o 


p 


p 


P 


96-56 


© 

i 


d 


d 


d 


d 

i 


d 

i 




Os 


r- 


i n 


o 




oo 




p 


p 


(N 


o 


p * 


p 


00-56 


d 

i 


d 

i 


d 

i 


i 


d 

i 


d 




r- 


o 


o 


00 


r-» 






(N 




Tf 


<N 


o 


o 


66-56 


d 

i 


d 

i 


d 


(N 

i 


d 


d 






r- 


Ov 


O 




o 




Tf 


i— < 


i— « 


p 


r- 


<N 


86-56 


d 

i 


d 

i 


d 

i 


m 

i 


d 


d 




00 


Tf 


<N 




ov 






p 


vq 


p 


« 


p 


(N 


L6-96 


d 

i 


d 

i 




• 


d 


d 




r- 


cn 




vo 


, 


m 




(N 


p 




p 


(N 


(N 


96-56 


d 

i 


d 

i 


d 


* 


d 


d 



u 

Cl 



00-56 



66-56 



86-56 



L6-96 



96-56 



00-56 



66-56 



86-56 



L6S6 



96-56 



<L> 

3 

o\ 

TD 

QJ 

-o 

c 

I 

c 

CL) 

O. 



U 

a 



>A — 

’cs 
> 
cd 



<D 



c A it 






C/5 






05 



(N 



4T5 

H 



o 

ERIC 



Appendix B 

Instrument Descriptions 



INTRODUCTION 

This appendix describes the development, characteristics and analyses of the instruments 
used in the summative evaluation of the SS&C project. The SS&C project is a teacher 
enhancement process and set of materials for science education in grades 9-12, but the 
SS&C project and this evaluation were funded by the National Science Foundation for 
only the ninth grade portion. The SS&C project was committed to developing activities 
that produced students that were better prepared according to the National Research 
Council's Science Education Standards. As a result, these content and teaching standards 
served as the guidelines for the development of the assessment instruments used in this 
evaluation (National Research Council, 1995). This means that the SS&C project was 
evaluated on how well the curriculum prepared students based on the standards not on the 
specific goals or objectives of SS&C. The seven science education content standards 
have specific goals and objectives for the various grade levels: K-4, 5-8 and 9-12. The 
content standards are: science as inquiry, life science, physical science, earth and space 
science, science and technology, science in social and personal perspectives and history 
and nature of science. 

The evaluation gathered data about students, teachers and schools at thirteen sites 
nationally over a three-year time frame and then followed on with five sites for another 
three years. During the first year of the evaluation effort data were collected from 
students similar to those who would be receiving the SS&C curriculum during the second 
year. During the subsequent years data were collected from SS&C students at the same 
schools where data were collected during the first year. Evaluation team members visited 
all sites in the fall and spring of all but one year when only a spring visit was made. Ten 
instruments were developed to assess student understanding of the content standards, 
student attitudes and motivation and the classroom environment in which these 
understandings were developed: a classroom observation schedule, principal, teacher and 
student interview protocols, a teacher questionnaire, a course content survey, a student 
questionnaire, a student hands-on full investigation test, a student laboratory performance 
test, and a student science literacy test. In addition to using these instruments to assess 




417 



the project, a case study of three teachers involved in the implementation was also 
conducted. 

The primary goal of the evaluation was to assess understanding of the content standards, 
therefore achievement was assessed in a variety of ways to help ensure validity of the 
findings. The students completed traditional multiple choice items, open-ended items, a 
series of hands-on laboratory stations, a hands-on full investigation and a written full 
investigation. Members of the evaluation team administered all of the assessment 
devices during site visits except the science literacy tests, which included the multiple 
choice and open-ended items. The literacy tests were mailed out to the sites for the 
teachers to administer. To maintain confidentiality the tests were counted into classroom 
sets with class lists before mailing. Teachers were asked to not make copies of the tests 
and to return all used and unused copies. Standardized administration protocols were 
developed and read to the students at the beginning of each testing situation to insure 
consistency of administration. 

Individual descriptions of each instrument follow. The student outcome measure 
instruments will be described first, followed by descriptions of those instruments 
designed to provide descriptive and environmental information. 



STUDENT OUTCOME MEASURES 

Science Literacy Test 

The purpose of the science literacy test was to assess comparison and SS&C student 
understanding of the seven NRC science content standards. It was critical that this test 
contained science items deemed appropriate by a national audience so the items for the 
science literacy test were selected from existing tests such as NAEP, SISS, and IAEP. 
The items for those tests had been developed through a rigorous consensus and review 
process and thus represented a national perspective on appropriateness. Additionally the 
national data available for some of the items could be used as a comparison for the results 
from this evaluation. A team of science educators went through all of the available items 
independently and rated them on their relevancy to the new NRC standards and their 
appropriateness for ninth graders. The independent ratings were combined and the team 
discussed any controversial items to determine the value of including them. Care was 



taken to select both open-ended and multiple choice questions and to not overlap on 
content. Outside science education experts reviewed all of the items in the final selection 
to confirm that the items were, indeed, consistent with the NRC standards. A draft 
version of the Science Literacy Test was pilot tested locally with 9th grade students at 
both a suburban and an urban school to assess the length, difficulty and readability of the 
test. 

Twenty-six items were selected for each of the seven NRC science content standards, 
except for the technology standard which had only 1 6 items because additional national 
assessment items that matched the standard could not be located. Of the 172 items 
administered to students, 12 were open-ended and 160 were multiple-choice. 
Approximately 40% of the items were higher level reasoning items requiring at least 
application level understanding as defined by Bloom (1956) and Madaus (1989). This 
was determined by having two independent raters rate each of the items after receiving 
training and becoming reliable raters on the Madaus definition. The items were used to 
construct two parallel forms of the science literacy test. Half of the students in each class 
took form A of the test while the other half took form B. Their teachers administered the 
test to students over two 45-minute class sessions near the end of the school year. 

Multiple-choice items on the science literacy test were scored as either correct or 
incorrect. All collected data were scored. Initially, standard errors were estimated using 
the bootstrap method because two parallel forms of the science literacy test were 
administered to students, (Efron & Tibshirani, 1993). The bootstrap method estimates 
the overall standard error of both forms of the test by taking the average of 200 randomly 
sampled standard errors. However, subsequent comparisons of the standard errors 
calculated using DESCRIPTIVES on SPSS showed that the calculated value of the 
standard error using ordinary methods falls in the middle of the distribution of a sample 
of 100 bootstrap estimates of the standard error. This result is consistent with the 
findings of other studies examining the utility of bootstrapping (Welch, Lawrenz, & 
Huffman, 1998) and has the additional advantage of remaining constant across multiple 
calculations, unlike the bootstrap estimates. Comparisons between years were made on 
the total score of all multiple-choice items and on the multiple -choice items for each of 
the seven content standards. Comparisons were calculated using z- scores. Charts were 
created to show changes over time on total score and effect size across sites and by site. 
Detailed tables support the data in the charts and present the statistical test data. Tables 
also include scores by site for each of the seven NRC Science Standards along with the 



total score. In the first SS&C year, the Cronbach alpha reliability for the multiple choice 
items was .89 for form A and .88 for form B. The reliabilities for each scale, for forms A 
and B of the multiple choice science literacy items respectively, were science as inquiry 
(.59 and .62), physical science (.66 and .67), life science (.64 and .62), earth and space 
science (.53 and .56), science and technology (.13 and .06), science in personal and social 
perspectives (.71 and .68), and history and nature of science (.48 and .38). 

The 12 open-ended items on the science literacy test were scored for all classes and all 
students in 1995. Subsequently, only those classes that were selected for observations at 
each site had their open-ended items scored. The open-ended questions were also 
selected from the previously mentioned existing national tests to match the NRC 
standards. The items were included on the same test forms as the multiple- choice items 
and, therefore, the numbers of students and administration procedures were the same. 

The items were split across the two forms of the test so each student only answered six 
items. The Cronbach alpha reliability of Form A was .67 and for Form B was .69. The 
evaluation team developed a coding scheme for the items based on the recommended 
scoring rubrics and the uniqueness of these students' responses. Team members 
individually scored some of the tests and then the team discussed their coding to verify 
interpretations. This coding was then applied to the full set of tests. For final scoring the 
codes were collapsed into categories of completely correct (2 points), partially correct (1 
point), or incorrect (0 points). The maximum possible on the open-ended items was 12 
points, and the percent correct was calculated by dividing the actual umber of points 
received by 12 and multiplying by 100. Four members of the evaluation team scored the 
complete set of items; two members scored the items from the comparison year and two 
different members scored the items from the first SS&C year. Intra-rater and inter-rater 
reliabilities for the comparison year were assessed based on a random sample of 20 form 
A tests and 20 form B tests. For the comparison year the average intra-rater reliability for 
the two scorers was 93%, and the inter-rater reliability was 95%. The same procedure 
was used to calculate reliabilities for the two scorers in the first SS&C year. For the first 
SS&C year the average intra-rater reliability was 93%, and the inter-rater reliability was 
91%. The average inter-rater reliability between the comparison and each SS&C year 
was assessed by having the SS&C year scorers rescore a random sample of 20 tests of 
form A and 20 tests of form B from the comparison year. For the second through fifth 
SS&C years, respectively, the average intrar rater reliabilities were 93%, 94%, 94%, and 
97%. The average inter-rater reliabilities between the comparison and each SS&C year 



were 88%, 92%, 93%, and 96% for the second, third, fourth, and fifth SS&C years, 
respectively. 

Comparisons using z- statistics and effect size calculations were conducted for each year 
in contrast to the comparison year. These data are presented in cross site and by site 
tables. Both mean scores and effect sizes over the years are also presented in charts in 
the body of the report in both the cross site and by site sections. 

Lab Skills PerformanceTest 

The purpose of the hands-on lab skills test was to determine if there were any differences 
in the laboratory skills of the comparison and SS&C year students. As with the science 
literacy test, in order to assure national acceptance of the items used, the instrument was 
developed from already existing and validated items. Each station was selected from 
existing national performance tests including the NAEP, SISS, and the International 
Assessment of Educational Progress, the Assessment of Performance Unit (APU, 1981) 
(Welford, Harlen, Schofield, 1985) and the British Columbia Science Performance Test 
(Bartley, Carlisle & Erickson, 1993). The five stations included determining density, 
chemical testing of starch and sugar solutions, using a microscope, testing and identifying 
rocks, and using measuring instruments. The density station asked students to use lab 
equipment to find the mass, volume, and density of a sinker. The indicators station asked 
students to conduct a test using indicators to determine which solutions contained sugar, 
starch, and neither sugar nor starch. The microscope station asked students to draw what 
was a slide at 10X and higher magnifications and then to determine which threads were 
on top of others on a second slide using the microscope. The rocks station asked students 
to identify three rocks using physical properties and simple chemical tests. The 
instruments station asked students to conduct laboratory measurements (volume, 
temperature, length, mass) using lab equipment. A draft version of the lab skills 
performance test and the full investigation described below were pilot tested locally with 
9th grade students at both a suburban and an urban school to assess the length and 
difficulty of the items, along with appropriateness of equipment and presentation. 

Members of the evaluation team administered the stations during the spring site visits and 
all of the equipment except for triple beam balances and microscopes were provided by 
the evaluation team to insure consistency of the test administration. Because of the time 
needed to administer the practical tests, only random sets of students at each of the 
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thirteen school sites were selected to answer these questions. At each school site, the 
students who took the lab skills performance test were randomly selected from the three 
classes that were observed resulting in approximately 80-120 students completing the test 
each year. Students who did not take the lab skills performance test were asked to 
complete the student questionnaire described below. The lab stations were set up in a 
classroom and the selected students rotated through the stations spending seven minutes 
working on each one. 

Each of the five stations was scored using the scoring scheme devised by the developers 
of that test. These "raw" scores were then rescaled so that each variable used the same 
scale and thus contributed equally to the calculation of each station score and the total 
score. For example, the variable MVALUE (Mass Value) was originally scored on a 4- 
point scale (4 = Completely correct, 0 = Completely incorrect). It was converted to a 
five-point scale with the command: 

compute MVAL = ((MVALUE/4) * 5) 

This was done for each variable, and the variables were summed to get a station score 
using the SUM function in SPSS. Each variable was then divided by 5 and multiplied by 
1 00 to get a percent score. The total score for each station was also changed to a percent 
score. Finally, the station totals were added up, divided by the total number of rescaled 
points (165) and multiplied by 100 to get the overall percent score for the test. These 
percent scores for the stations and the overall test, with accompanying standard errors, 
are the primary numbers used in reporting this data. The total lab skill scores and the 
scores for each of the stations for each year of this study were compared to the scores in 
the comparison year using z-statistics. The scores and statistical tests for total score are 
presented in the cross site Tables and scores and statistical tests for total score and each 
station are presented in the by site Tables. Charts showing the changes in total score and 
effect sizes over time are presented in the body of the report in both the by site and cross 
site sections. 

The intra-rater reliability of the scoring was assessed by re-scoring a random sample of 
20 tests for each year. The 20 tests from the comparison year were also scored by 
another member of the research team to assess inter-rater reliability. The mean intra- rater 
reliability over both the comparison and first SS&C year was 93%, and the inter-rater 
reliability for the comparison year was 91%. The mean inter-rater reliabilities for the 
SS&C years ranged from 88% to 97%. The mean intra-rater reliabilities for the second 
through fifth SS&C years ranged from 91%-97%. Cronbach alpha reliability for the lab 



skills test in the first SS&C year was 84%. The reliabilities for each station were .85 for 
density, .90 for sugar and starch indicators, .67 for microscope, .85 for rock 
identification, and .48 for instruments. 

Full Investigation Test 

The purpose of the full investigation was to demonstrate student ability to design an 
experiment as proposed in the NRC inquiry standard. The hands-on version then asked 
the student to conduct the experiment and record the results. Both the written and hands- 
on full investigation tests were based on the classic “bugs test” developed by the 
Assessment of Performance Unit in the United Kingdom (APU, 1981). The test asked 
students to design an experiment to determine which of the following environments 
isopods prefer: light and dry, light and wet, dark and dry, or dark and wet. A member of 
the evaluation team scored both the hands-on and written versions of the test. Inter-rater 
and intra-rater reliability of the scoring of the hands-on full investigation test was 
checked by re-scoring a random sample of 20 tests from both the comparison year and 
the first SS&C year. For the comparison year the inter-rater reliability was 85%, and the 
average intra- rater reliability was 91%. For the first SS&C year the Cronbach alpha 
reliability for the hands-on full investigation test was .75. Inter-rater and intra-rater 
reliability of the scoring of the written full investigation test was checked by re-scoring a 
random sample of 20 written tests from both the comparison year and the SS&C year. 

For the comparison year, the inter-rater reliability was 85%, and the average intra-rater 
reliability was 92%. The individual who received this 92% intra-rater reliability 
subsequently scored both the written and hands-on full investigations for the six years of 
the project. The Cronbach alpha reliability for the written full investigation test in the 
first SS&C year was .88. 

Each year two students were randomly selected to complete this assessment from each of 
the three classes observed at each site. The students were given the necessary 
experimental apparatus and asked to draw their experimental set-up, describe it in writing 
and state their conclusions. Isopods, and various types of equipment like timers, sponges, 
black paper, containers, etc. were provided. A written version of this test (see the 
description of the student questionnaire below) was administered to students who did not 
take the hands-on test. The written version was identical to the hands-on test, except 
students did not have materials and were instead asked to imagine and describe how they 
would perform the experiment. 



The hands-on and written versions of the experimental design tests were scored in an 
identical manner using the APU scoring rubric that focused on aspects of the design and 
implementation of experiments. Five aspects of conducting an experiment were scored: 

1) the number of environmental conditions presented to the isopods, 2) the extent to 
which access to different conditions was controlled, 3) the extent to which the amount 
(i.e. the area or volume) of each environmental condition was controlled, 4) the primary 
method of measurement, and 5) the number of isopods used to conduct the experiment. 
Each of the five parts of the score were equally weighted and placed on a scale from 0 to 
10. This was accomplished by using COMPUTE on SPSS as follows: 

COMPUTE NumberP = (Number/5)* 10. 

A total score was calculated by adding the five parts of the score thus creating a score 
that ranged from 0 to 50. The percent correct for each subscore was calculated by 
dividing the weighted score, as calculated above, by 10 and multiplying by 100. The 
overall percent correct was obtained by dividing the total by 50 and multiplying by 100. 
The percent correct for each scoring category and the total score are included in the by 
site Tables along with z- tests results comparing the scores to the comparison year. Charts 
were created to show change over time on hands-on and written full investigation total 
scores and effect sizes across sites and by site. 

Student Questionnaire 



The purpose of the student questionnaire was to assess both students' perceptions of the 
learning environment as described in the NRC standards and students' attitudes and 
motivation. The questionnaire was 12 pages long and took about 30 minutes to complete. 
All of the appropriate ninth graders at each site except those selected for the hands-on 
testing completed the questionnaire. Responses to the items were directly inputted from 
the questionnaire and average ratings were calculated and compared to the comparison 
year using z- tests. 

The bulk of the questionnaire was a six-scale learning environment inventory containing 
37 items. The 5-point Likert scale items measured six aspects of the learning 
environment: 1) involvement (6 items), 2) difficulty (6 items), 3) relevancy (7 items), 4) 
critical voice (7 items), 5) experimental design (5 items), and 6) sequence (4 items). The 
involvement scale assessed students' perceived personal involvement in the classroom. 
The difficulty scale measured the students' perception of the academic difficulty in their 



classroom. The relevancy scale concerned the perceived relevance of school science to 
students’ out-of-school experiences. The critical voice scale assessed the extent to which 
students believe it is acceptable and beneficial to question a teacher’s pedagogical plans 
and methods (Taylor, Fraser & White, 1994). The experimental design scale examined 
the students' perception of the degree of open-endedness in the design of the experiments 
that were conducted in the classroom. The sequence scale measured the perceived 
sequence of classroom instruction, for example, whether or not the teacher used hands-on 
activities before lecturing to the class. All items were written in the personal form as 
recommended by Fraser and Tobin (1991), in order to more accurately elicit students' 
perceptions of the classroom learning environment. The personal relevance and critical 
voice scales were taken intact from Taylor, Fraser and White's (1994) Constructivist 
Learning Environment Scale. The other four scales, experiment design, student 
involvement, sequence and difficulty, and were developed by the evaluation team. The 
development consisted of selection, modification or invention of items based on other 
environment inventories. This was followed by a series of two field tests with samples of 
both urban and suburban 9th graders, reliability estimations and factor analyses. Based 
upon the results of the reliability analysis and the factor analysis, modifications were 
made to the items and the new items were field tested again with another sample of 
students. All items were also checked for readability and interpretation before field- 
testing. The Cronbach alpha reliability for each scale on the learning environment 
instrument in the first SS&C year was as follows: involvement, .70; difficulty, .79; 
relevancy, .76; critical voice, .80; experimental design, .55 and sequence, .61. 

In addition to the learning environment items, the student questionnaire also requested 
demographic information, contained 24 items related to student s' attitudes toward 
science, and contained the written version of the full investigation test. In the attitude 
section students were asked what science activities they engaged in outside of school (10 
Yes/No items), what science courses they planned to take in the future, if they are 
motivated by science and their experiences in their science class (8 Yes/No items) and 
their perceptions of the class as fun, interesting, relevant, etc. (6 Likert items). 

The variables in the LEI- Student Questionnaire are laid out in a relatively straightforward 
manner, but breakdown of the variables is somewhat complex. A variety of different 
types of items are being used and each type has a slightly different procedure for analysis. 




4.25 



The first 37 items of the questionnaire comprise the Learning Environment Inventory. 

The subscales of the LEI are computed using the MEAN procedure in the SPSS control 
file and include Difficulty, Relevance, Critical Voice, Involvement, Sequencing, and 
Experimenting. The means and standard errors of these subscales are reported along with 
the results of t-tests comparing the scores to the comparison year are presented in the by 
site Tables, and charts showing change over the years are in the by site section of the 
report. 

The next set of variables includes items that ask about science activities in which the 
students participate outside of class. These 10 variables have been recoded so that 100 = 
Yes and 0 = No. Thus, these variables are reported as the percentage of students who 
indicated that they participated in each activity. These data and the results of z- tests 
comparing the later years of the comparison year are presented in the by site Tables. 

A similar recoding procedure was used on q52a to q56, the eight variables that addressed 
the student's intention to continue taking science classes and science motivation. The 
analysis of these variables represents the percent of students responding "Yes" to each 
item. These data and the results of the z-tests comparing the later years to the 
comparison year are presented in the by site Tables. Subsequent analyses are described 
below. 

Items 57 to 62 ask questions about the student’s attitudes about science. These six items 
were initially reported individually on the 5-point Likert scale. This means, standard 
errors and t-tests comparing the later years to the comparison year are presented in the by 
site Tables, and the mean responses for each item are presented in the by site charts. 

The last two years of the project, the attitude variables were also recoded into a yes/no 
format and combined with the items addressing inclination to continue studying science 
(with the exception of q52a, Do you plan to take science in 10 th grade?, because all 
students in the sample were required by law to take science in 1 0 th grade so there was no 
variability on that item) and science motivation, in a single Inclination, Motivation, and 
Attitudes scale. Responses indicating disagreement with the attitude items (i.e. a 1 or a 2) 
were recoded as No = 0 and responses indicating agreement (i.e. a 4 or a 5) were recoded 
as a Yes = 100. Responses that indicated no opinion on the attitude items (i.e. a 3) were 
coded as system missing. Since this resulted in a sizable number of missing variables, 
the new scale was calculated "by hand", not by using MEAN in SPSS. The formula was 



as follows: (q52b+ q52c+ q52d+ q53+ q54+ q55+ q56+ newq57+ 

newq58+newq59+newq60+newq6 1 +newq62)/l 3 . The results of this combined score and z-tests 
comparing the later years to the comparison year are presented in the cross site Tables. 
The scores over the years are displayed in charts in the body of the report in the cross site 
and by site sections. 

Items 63 to 72 inquired about the kinds and frequency of various activities in class. 

These items asked students to estimate the frequency of various teaching techniques in 
their science classrooms. For the first four years of the project these variables were 
reported just as they were, calculating the mean response for each item. The last two 
years, these items were factor- analyzed and regrouped based on the results of that 
analysis. The data about individual items included in this report were those items 
included in one of the new groups. The final regrouping of the items is as follows: 



New grouping 


Items included 


Inquiry 


q74 - Demonstration of a scientific principle 
q76 - Ask students to suggest hypotheses 
q77 - Ask students to interpret data 
q78 - Relate current topic to previous work 


Groups 


q63 - Do experiments with other students 

q69 - Work in groups 

q71 - Share experiment results 


Traditional 


q65 - Read articles about science 
q66 - Do an oral or written report 
q67 - Read other materials 


Other 


q68 - Watch films, slides, or videos 
q70 - Go outside for class instruction 



The above scales were calculated using MEAN on SPSS. The results of this 
recombination were reported in two ways: as the mean score of the scale on the original 
5-point Likert scale and as the percent of time spent in each category. To obtain this 
number, the four scales were added together to obtain a "total score" of sorts. Then, each 
category score was divided by this new total and multiplied by 100 to obtain an estimate 
of the percent of time spent in each type of activity. The mean and standard error and t- 
test comparisons for the later years with the comparison year for the totals and by items 
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are presented in the by site Tables. The total scores and statistical tests for frequency and 
percent time are also presented in the cross site Tables. Charts for percent time over the 
years are in the by site section of the report. 

The last two pages of the instrument are the written form of the Full Investigation and are 
coded separately from the rest of the questionnaire. See the section describing the Full 
Investigation for further details. 
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DESCRIPTIVE MEASURES 



Classroom Observations 



Classroom observations were conducted to verify implementation of the SS&C 
curriculum and to document the behaviors in an actual classroom setting. In the 
comparison and the SS&C years each site was visited twice, once in the fall and again in 
the spring. The only exception to this occurred during the 1997-98 school year. No site 
visits were made in the fall of 1997 due to funding constraints. During each visit three 
classes, taught by three different teachers if possible, were observed. Classroom 
observations typically lasted for 45-50 minutes, depending upon the length of the class 
period. Given the limited number of classroom observations, these were viewed as 
supporting documentation, rather than as a primary data source. 

The classroom observation form was developed from the descriptions of ideal science 
classrooms in the standards. The observation schedule was two pages long and had forty 
different items documenting several aspects of the learning environment including the 
following information: student involvement, student and teacher questioning behavior, 
teacher rapport, cooperative group work as defined by Johnson and Johnson (1991), type 
of laboratory work as based on Schwab's levels of involvement, relevancy of content to 
the "real world", and critical voice of the students (i.e., the extent to which students 
believe it is acceptable and beneficial to question the teacher’s methods and knowledge 
claims). The form also included items describing the setting and the equipment available. 
Throughout each observation period, the observers documented the most common 
form(s) of instruction that occurred during every 5- minute interval. In order to ensure 
consistency of use of the instrument, all of the evaluation team observed several classes 
locally using the schedule before it was finalized to achieve consensus of opinion on 
categorizations. As a final check after the first use of the schedule at the sites, the 
evaluation team members discussed what had been observed and decided how to best 
represent that on the schedule. As an additional attempt at consistency, the same 
observers conducted the site visits all years whenever possible. 

The observation data regarding the specific activities or teaching strategies being used in 
the class during each five-minute interval was coded into categories. For the data 
collected from Fall 1994 to Spring 1997, these categories included Lecture, Small Group 
Discussion, Demonstration, Lecture with Discussion, Administrative Tasks, Cooperative 



Learning, Class Discussion, Text Seat Work, Teacher Interacting with Students, Hands- 
on Activity /Materials, Worksheet Work, and Other. Data from Spring 1998 onwards 
included two additional categories: Reading Seat Work and Lab Write-up. 

The SPSS control file read these data as both a six-digit code per interval and as three 
separate two-digit codes. That is, in the SPSS data list file used to create the working 
system file, the following code would have appeared to describe the data collected in the 
first five-minute observation period: OBSIO 25-30, OBS10A 25-26, OBS10B 27-28, 
OBSIOC 29-30. Rather than focus on the individual codes, it was decided that the six- 
digit codes would be collapsed into five broad categories of class activities. Each six- 
digit code was assigned to only one category. The original data were therefore recoded 
as follows: 



Lecture, Worksheet Work, Text Seat Work and 
Reading Seat Work alone, in combination or with 
other codes 


Teacher-Centered 


Lecture with Discussion, Class Discussion, 
Demonstration alone, in combination or with other 
codes 


Both Teacher- and Student -Centered 


Hands-on Activities/Materials, Small Group 
Discussion, Lab Write-up alone, in combination 
or with other codes 


Student-Centered 


Administrative alone or with other codes 


Administrative 


Other activities 


Other 



The recoding process took several steps. The first step was to list the possible values for 
each six-digit code. Then, the codes were reviewed and were assigned by hand to one of 
the five categories. For example, if the six-digit code was 041102 (Hands-on Activity, 
Teacher Interacting with Students, and Lecture with Discussion) the variable was recoded 
as 2, Both Teacher and Student-Centered. In SPSS, this recoding resulted in the creation 
of the variables “newobsl0...newobs21”. Then, using COUNT in SPSS, the five 
categories were created as follows: 

COUNT TeaCen = NewobslO thru NewObs21(l). 

COUNT STCen = NewobslO thru NewObs21(2). 

COUNT StuCen = NewobslO thru NewObs21(3). 



COUNT Admin = NewobslO thru NewObs21(4). 
COUNT Other = NewobslO thru NewObs21(5). 



Finally, the DESCRIPTIVES command with the SUM subcommand was used to get the 
total number of responses that fall into each category. The percentages of responses in 
each category were calculated by hand, as were the chi-square tests comparing each 
SS&C year to the comparison year. These data are presented in the by site Tables and 
charts are included in the by site section of the report. 

In the final year of the project, these data were also reported as a continuous variable 
showing the percent of time spent in Teacher-Centered, Both Teacher- and Student- 
Centered and Student- Centered activities. This was done by summing the totals of all 
five categories to obtain an overall "total score" of sorts then dividing the totals for each 
of the three categories mentioned above (Teacher-Centered, Both Teacher- and Student- 
Centered, and Student- Centered) by this overall total and multiplying by 100. The 
MEAN procedure in SPSS was then used to calculate the average percent of time spent in 
each category for each site. These data are presented in charts in the cross site section of 
the report and are included in the cross site Tables. 

The rest of the data addressing the classroom environment, the ratings of teacher rapport, 
student involvement, critical voice, level of open-endedness, relevancy, and type and 
number of teacher and student questions, is largely categorical data of the “Always, 
Sometimes, Never” or “Yes, No” variety. It was summarized using CROSSTABS and 
the chi-square test was used to compare data across years. Only data on observation 
ratings of the level of student involvement and type of instruction were included in this 
report. These data are presented in the by site Tables and discussed in the text. 

Principal, Teacher and Student Interviews 

The interviews were designed to corroborate data from the other instruments and to allow 
participants the opportunity to describe things in their own way. Interviews were 
conducted one-on-one with members of the evaluation team during the site visits. 
Interview questions were matched to questions asked of the students on other instruments 
and those used on the observation schedule to facilitate cross verification. The principals 
were interviewed once in the fall. The teachers were interviewed in both the spring and 
the fall. The three teachers at each site whose classes had been selected for observation 



were interviewed using the long form while other ninth grade teachers were interviewed 
using the short form. After the third year of the project the teacher interview forms were 
changed to reflect the new longitudinal perspective of the evaluation. One student in 
each of the observed classes was interviewed during the spring site visit. 

The school principal interview was designed to obtain a general description of the school 
and community, and to elicit the principal's perceptions of the stability of their school 
populations as a check on the validity of between-year comparisons. The protocol was 
about one page long, included 20 questions and took approximately 1 5 minutes to 
administer. Data from four items addressing sources of potential problems are the only 
items for which there is numerical data. 



Student interviews were conducted to verify student perception of the learning 
environment in the course they were taking and their attitudes toward science. The 
interview protocol was two pages long, included fourteen questions and took 
approximately 1 5 minutes to administer. Responses to the eight items which produced 
numerical data included both Yes/No responses and A lot/Some/Never responses. 

Teacher interviews were designed to obtain teachers' perceptions of their classroom 
activities and the learning environment in their classrooms. From Fall 1994 to Spring 
1998, two versions of the interview form were used. The long form interview was 
administered to the three observed teachers at each site, and the short form was used for 
any additional teachers. The long form asked questions specifically about the class that 
had been observed as well as questions about the teachers' approaches to teaching 
science. The long form was three pages long, contained 17 different questions and took 
about 30 minutes to administer. The short form was identical to the long form except that 
ten of the more specific questions were removed. 

In the 1998, 1999 and 2000 school years, a different version of the interview form was 
used. Teachers were asked about the support for their courses, the vision the teacher had 
for science education, and how decisions got made at their schools. In addition, teachers 
were asked to estimate the percent of lab activities that fall into one of four categories 
that varied in the relative amounts of student and teacher control, to describe general 
patterns they remembered from each year of the project, and to rate how well the 
teacher's classes were aligned with the science content standards. No data from the 



interviews are presented in the Tables or charts. The data were used to inform the 
qualitative impressions. 

Teacher Questionnaire 



The teacher questionnaire requested demographic information, teachers' ratings of how 
often they used various instructional techniques, and teachers’ perceptions of the learning 
environment of their classrooms. From 1995 to 1997, the questionnaire was four pages 
and had 34 five option Likert scale rating questions addressing instructional techniques 
and learning environment, and 13 additional items which teachers used to indicate the 
amount of emphasis they placed on various topics throughout the year. The items that 
asked teachers to indicate how often particular instructional techniques occur were 
matched to similar items on the Student Questionnaire and the classroom observations. 
Here, as on the Student Questionnaire, the instructional techniques were factor analyzed 
and the results were used to regroup these items into two new variables. The resulting 
groups are listed below: 



New grouping 


Items included 


Inquiry 


ql - Do experiments with other students 
ql6 - Ask for reasons for results of 
experiments 

ql 7 - Ask students to suggest hypotheses 
q 1 8 - Ask students to interpret data 


Traditional 


q3 - Read articles on science 
q4 - Do an oral or written report 
q5 - Use supplementary science materials 
for reading 



These are the categories used to report the data in the 1995-2000 report, using the MEAN 
function on SPSS to calculate the variables. The results of this recombination were 
reported as the mean score of the scale on the original five point Likert scale for each 
variable. Measures and standard errors for the total scores and each item along with t- 
tests comparing the later years to the comparison year are presented in the by site Tables. 
Total scores are presented in the by site charts. 



In addition to these preceding items, from Spring 1998 onward, the questionnaire also 
included items that asked the teachers to estimate the percent of time they spend in 
various kinds of class activities. It also asked teachers to estimate the amount of time 
they spent on specific individual, group, and whole class activities. Individual activities 
included hands-on activities, lab write ups, textbook and other reading seat work, 
worksheet work, journaling, computer work and other. The group activities included 
hands-on activity, small group discussion, lab write ups, worksheet work and other. 
Whole class activities included lecture, class discussion and demonstrations These data 
were summarized and mean scores and standard errors are presented for the three years in 
the by site Tables. 

Course Content Survey 

The course content survey was used to document the topics taught during the comparison 
and SS&C years. One teacher at each school was asked to list the topics they taught in 
their science classes and to estimate the number of weeks spent on each topic. In 
addition, teachers were asked to examine a list of twenty-eight topics from the NRC 
standards and estimate the number of weeks spent on each topic. The survey was three 
pages long and was mailed out for the teachers to complete with the literacy tests near the 
end of the school year. The first section of the survey was analyzed by assigning each 
topic listed by the teacher to one of seven categories, Chemistry, Biology, Earth and 
Space Science, Physics, Measurement, Science and Society, and Miscellaneous. The 
numbers of weeks spent on each topic indicated were then summed up for each category. 
The data for this part of the survey was then reported as the number of weeks spent on 
each category at each site. The number of weeks and percent of total class time are 
presented in the by site Tables. 

The second part of the course content survey was analyzed by grouping the 28 topics 
listed into seven categories according the NRC standards: physics, life science, earth and 
space science, science and technology, science in personal and social perspectives, 
inquiry, and the history and nature of science. 
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